Method and system of wireless data transmission for virtual or augmented reality head mounted displays

ABSTRACT

A system, article, and method of wireless data transmission for virtual or augmented reality head mounted displays.

BACKGROUND

Head mounted displays (HMDs) are worn over the eyes and present imagesto a user wearing the HMD to provide the user a point of view (POV) in avirtual or augmented reality (or world). The HMD is conventionallyconnected by one or more wires to an image processing base, such as agame console, computer, or dock. In order to provide real-time images onthe HMD, sensors on the HMD transmit data indicating the 3D location andorientation of the HMD to the image processing device or base, and thebase then transmits the image to the HMD over the wires. Due to theconstant motion of the HMD, the location and orientation of the HMD mustbe updated constantly with new images so that the images displayed onthe HMD match the motion of the HMD in near real-time so that the userfeels as if they are in the virtual or augmented world. This requires avery large amount of computations usually limiting the HMD-to-baseconnection as a wired connection. Such a wired connection can be veryrestrictive by limiting the range of motion of a user wearing the HMD,and can be cumbersome when the need arises to frequently move the wireout of the way as the user moves.

Short range or personal area networks such as those based on WirelessGigabit Alliance (WiGig) certification program standards have beentried. A WiGig system uses beamforming to direct the data transmissionto the HMD. The WiGig system uses a radio-based search by signalstrength to direct the beam in the correct direction toward the HMD.Such a system, however, often is too slow and cannot perform the searchwith sufficient speed and frequency to provide smooth video with highlyaccurate, near real-time perspectives to the HMD, especially when theuser is in constant motion. This can result in a significant reductionin video quality where the perspective of the images do not match thereal-time position of the HMD or even complete signal loss resulting ina blank or static image and therefore, the system may become unusable.

DESCRIPTION OF THE FIGURES

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1 is a schematic diagram of a virtual reality or augmented reality(VR/AR) head mounted display (HMD) system according to theimplementations provided herein;

FIG. 2 is a schematic diagram of a system for generating a virtual oraugmented reality on an HMD according to the implementations providedherein;

FIG. 3 is a flow chart of a method of wireless data transmission forhead mounted displays according to the implementations herein;

FIG. 4 is a flow chart of a method of wireless data transmission forvirtual or augmented reality head mounted displays according to theimplementations herein;

FIG. 5 is a schematic diagram of one example VR/AR head mounted displaysystem arranged to perform an initialization process according to theimplementations provided herein;

FIG. 6 is a schematic diagram of the example VR/AR head mounted displaysystem arranged to perform a run-time process according to theimplementations provided herein;

FIG. 7 is a detailed flow chart of a run-time method of wireless datatransmission for virtual or augmented reality head mounted displaysaccording to the implementations herein;

FIG. 8 is a detailed flow chart of another run-time method of wirelessdata transmission for virtual or augmented reality head mounted displaysaccording to the implementations herein;

FIG. 9 is a diagram of an operation of an example system describedherein;

FIG. 10 is an illustrative diagram of an example system;

FIG. 11 is an illustrative diagram of another example system; and

FIG. 12 illustrates another example device, all arranged in accordancewith at least some implementations of the present disclosure.

DETAILED DESCRIPTION

One or more implementations are now described with reference to theenclosed figures. While specific configurations and arrangements arediscussed, it should be understood that this is done for illustrativepurposes only. Persons skilled in the relevant art will recognize thatother configurations and arrangements may be employed without departingfrom the spirit and scope of the description. It will be apparent tothose skilled in the relevant art that techniques and/or arrangementsdescribed herein also may be employed in a variety of other systems andapplications other than what is described herein.

While the following description sets forth various implementations thatmay be manifested in architectures such as system-on-a-chip (SoC)architectures for example, implementation of the techniques and/orarrangements described herein are not restricted to particulararchitectures and/or computing systems and may be implemented by anyarchitecture and/or computing system for similar purposes. For instance,various architectures employing, for example, multiple integratedcircuit (IC) chips and/or packages, and/or various computing devicesand/or consumer electronic (CE) devices such as imaging devices, digitalcameras, smart phones, webcams, video cameras, video game panels orconsoles, set top boxes, and so forth, may implement the techniquesand/or arrangements described herein being, or being remotely connectedto, a head mounted display or other wireless devices to direct atransmission beam to the device, such as a phone, tablet, etc., as longas the device was configured with a low latency, high precisioncapability describing the position of the device at any instance.Further, while the following description may set forth numerous specificdetails such as logic implementations, types and interrelationships ofsystem components, logic partitioning/integration choices, and so forth,claimed subject matter may be practiced without such specific details.In other instances, some material such as, for example, controlstructures and full software instruction sequences, may not be shown indetail in order not to obscure the material disclosed herein. Thematerial disclosed herein may be implemented in hardware, firmware,software, or any combination of these

The material disclosed herein also may be implemented as instructionsstored on a machine-readable medium or memory, which may be read andexecuted by one or more processors. A machine-readable medium mayinclude any medium and/or mechanism for storing or transmittinginformation in a form readable by a machine (for example, a computingdevice). For example, a machine-readable medium may include read-onlymemory (ROM); random access memory (RAM); magnetic disk storage media;optical storage media; flash memory devices; electrical, optical,acoustical or other forms of propagated signals (e.g., carrier waves,infrared signals, digital signals, and so forth), and others. In anotherform, a non-transitory article, such as a non-transitory computerreadable medium, may be used with any of the examples mentioned above orother examples except that it does not include a transitory signal perse. It does include those elements other than a signal per se that mayhold data temporarily in a “transitory” fashion such as RAM and soforth.

References in the specification to “one implementation”, “animplementation”, “an example implementation”, and so forth, indicatethat the implementation described may include a particular feature,structure, or characteristic, but every implementation may notnecessarily include the particular feature, structure, orcharacteristic. Moreover, such phrases are not necessarily referring tothe same implementation. Further, when a particular feature, structure,or characteristic is described in connection with an implementation, itis submitted that it is within the knowledge of one skilled in the artto affect such feature, structure, or characteristic in connection withother implementations whether or not explicitly described herein.

Systems, articles, and methods of wireless data transmission for virtualor augmented reality head mounted displays.

As mentioned above, it is desirable to improve VR/AR HMD systems byeliminating the data transmission cable that tethers the HMD to the base(such as a host personal computer (PC)) in order to provide the userwearing the HMD more freedom of movement. Also as mentioned, the use ofwireless networks with large transmission bandwidths, such as WiGig forexample, can be used to transmit data between the HMD and the base. TheHMD (or AR/VR system) has sensors to indicate the position andorientation of the HMD and that is transmitted to the base, while thebase performs calculations based on the positional information generatedon the HMD (or AR/VR system) to determine how to render the imagesdisplayed on the HMD. The images are then transmitted to the HMD. Thisallows for a cable free VR or AR experience enabling full freedom ofmotion within a local environment

WiGig, however, uses beamforming to focus the signal transmissiondirectly from the transmitter (or source) to the receiver (or sink) inboth directions between the HMD and the base (e.g., both the HMD and thebase have both a beamforming transmitter and receiver). Under theconventional application of WiGig, the base is connected to a staticdock station and occasional movement could be tolerated through thistechnique. In a VR application, however, constant motion at the HMDcreates a continuous need to refocus (or steer) the directed signal inthe direction of a receiver on one of the devices (the HMD or the base),and/or to regenerate the incident transmission angle of the beam. Thetransmission angle is the angle of the beam relative to the normalizedangle with respect to the plane formed by a WiGig antenna array.

The process of steering or regenerating the beam often involves analysisof the signal strength on the receiver as the beam adapts to determineif an optimal direction is being achieved. In most cases, a search forthe best beam direction is determined by scanning a certain area nearthe last position of the HMD and in a sweeping motion searching for thestrongest signal. The conventional implementation in the WiGig solution,however, does not consistently maintain a sufficiently accurate beam dueto the latency caused by this steering operation, and therefor resultsin stuttering (or loss of video smoothness) and lost information as thestream of visual content is sent to the HMD. This results in a mismatchbetween the image that is shown to the user wearing the HMD and themotion of the HMD. The impact to the user experience can be extremelydisorienting where the image provides a view to the user of a stationaryscene as if the user is not moving his head while the user knows orfeels he is moving his head with the HMD, or vice-versa where the HMD isshowing a scene in motion while the user is not moving. This poor videosmoothness can render the HMD unusable.

To resolve the issues mentioned above, the present method and systempropose to use the sensor data from the HMD (or AR/VR system) to providereal-time HMD position information to automatically focus the signaltransmission beam whether from the transmitter at the base to thereceiver on the HMD, or the transmitter at the HMD to the receiver atthe base. This may include determining a precise initial beam position(or angle) aimed at a start HMD position where the position of the HMDis determined relative to the base. In many conventional sensor-basedsystems, the sensors may establish a 3D space or model in which the HMDmay be moved from position to position, but there is typically no needto determine the position of the base in such a system. During thisinitialization stage, radio-based searches still may be used.Thereafter, however, sensor data alone, without the use of a radio-basedsearch for the HMD determines a change in HMD position relative to thestart HMD position (or some other prior precise location), whichindirectly positions the HMD relative to the base so that a beamposition from the base to the current HMD position can be determined totransmit images from the base to the HMD. Even with the additional timewaiting for the sensor transmission, the total time from the motion ofthe HMD to the transmission of image data along the correct direct lineto the HMD is still significantly rescued compared to a conventionalsystem that uses radio-based searching during the run-time use of theHMD. The resulting reduction in latency permits image data to betransmitted to the HMD and continuously displayed in real-time ornear-real time as perceived by the user wearing the HMD therebysignificantly improving the real-time image displaying function of theHMD system, and providing a high quality experience for the user.

When the HMD is the transmitter, either the HMD or the base may computethe beam position to transmit sensor data from the HMD to the base. Thismay be accomplished by determining an incident angle that is the changein beam position from the initial beam position to a desired currentbeam position extending from the current HMD position to the base. Theincident angle includes a position component angle that factors thechange in translational position of the HMD from the reference positionto the current position, and an orientation component angle that factorsthe rotational change about the HMDs three axes and from the orientationof the HMD at the reference position to the orientation of the HMD atthe current position. A number of variations are included herein whereone or both of the beam positions are computed by the HMD, base, otherelectronic device communicating with the HMD and/or base, or somecombination of these.

It will be appreciated that real-time (or more precisely, nearreal-time) refers to the amount of time that is still perceived to bereal-time (or sufficiently close to it for viewing images as intended)by a person. By one form, real-time herein may be about 5 ms or lessfrom receipt of the sensor data regarding a position and/or orientationof the HMD to the display of the image.

Referring to FIG. 1, a system 100 for displaying a virtual or augmentedreality shows a user 102 wearing an HMD 104 with a strap 108 around theuser's head to hold an HMD body in front of the user's eyes. The body106 holds one or more display screens 110 facing the user's eyes on theinside of the HMD body 106. By some example forms, the HMD body 106 mayhave sides to block light from entering an interior space between theeyes and the display screen. Other HMDs, such as smart glasses thatprovide augmented reality, may be open on the sides and may havetransparent glass at the location of the display(s). Two displays 110may be provided with one display for each eye, or alternatively, theremay be one single display provided or some other configuration. Thedisplays show the virtual or augmented reality to the user 102 so thatthe user is provided a personal point of view (POV) as if the user waswithin that displayed reality world.

The HMD may have an antenna 114 to receive and send radio signals forexample, and that provides the signals to a receiver/transmitter (ortransceiver) unit 116, which in turn may be wired or otherwisecommunicating with at least one display manager 118 and at least onesensor unit 112. The display manager 118 decodes image data and hascontrols for displaying the image data on the screen(s) 110. The sensorunit 112 may have one or more sensors for sensing the position and/ororientation of the HMD 104. The sensed position of the HMD may becoordinates in a 3D space (x, y, z). At first, this space is establishedby sensors to determine the translational positions of the HMD as theHMD moves in the 3D space. At this point, an HMD position may bedetermined relative to other HMD positions, but the position of the base120 in this sensed 3D space may not be established. As explained indetail below, further HMD position calculations using the sensedpositions may be used to determine the position of the HMD relative tothe base. This may or may not include a different, separate coordinatesystem where the base is set at the (0, 0, 0) position of the coordinatesystem for example. The details are provided below.

Otherwise, the sensor unit 112 also may include operations used todetermine the orientation of the HMD relative to a reference orientationsuch as a level, up-right position (or zero orientation) for examplewhere X, Y, Z axes of the HMD 104 are respectively parallel to x, y, z,axes of the 3D space. The orientation may be provided in (i, j, k)degrees where each of i, j, k refers to rotation about a respective axis(X, Y, Z) of the HMD. Other sensors for orientation and/or position ofthe HMD may be placed on a base 120 or other electronic devicecommunicating with the HMD or other locations external to the HMD and/orthe base in order to perform some or all of the beam positioncalculations described herein. The details are described below. Forconsistency, orientation of the HMD will be referred to when discussingthe tilt or angle of the HMD about its own axes XYZ, and position (orlocation) of the HMD will be referred to when discussing the translationof the HMD in the directions of the axes (xyz) and the location of ananchor point on the HMD in a 3D space. The beam position then mayinclude an angle alone (and may be referred to as the beam angle), startand end points which may be the anchor points on the HMD and base, orboth the angle and points.

The base 120 has a sensor analysis unit 124 that determines the positionof the HMD relative to the base, and provides instructions to abeamforming control 126 that controls an antenna 122 on the base and todirect the beam to the HMD 102. While not shown here, the HMD 102 mayhave its own sensor analysis unit and/or beamforming control unit todirect the beam to the base by controlling the antenna 114 in order tosend sensor data to the base. Alternatively, either the base or the HMDmay compute a different beam position to transmit at least sensor datafrom the HMD to the base. The details are provided below.

By one form, the HMD 104 transmits at least sensor data to the base 120while the base transmits at least images to the HMD 104 and based on, atleast in part, the sensor data. The transmission may be conducted over aradio-based steerable transmission beam of a high bandwidth, low latencynetwork, such as Wireless Gigabit Alliance (WiGig) certification programstandard IEEE 802.11ad by one example. The centerline of a steerablebeam 128 is shown at a beam position 134, and a dashed sensor datatransmission line 132 transmitting sensor data from the HMD 104 to thebase 120 and a dashed image data transmission line 130 transmittingimage data from the base to the HMD represent these transmissionsoccurring over or along the beam 128 at the beam position 134. By oneform, the beam positions are directed between the antennas 114 and 122of the base and HMD respectively.

The base 120 may be, or may be part of, any electronic device thatprovides the functions of the base described herein. This may include adedicated game console that is provided solely for operation of the HMDor may include game or HMD consoles with multiple functions such asthose that display a game on a television or monitor, or other boxessuch as a TV or set-top box (e.g., a cable or satellite box), and soforth. Otherwise, the base may be a computer such as a PC desktop,laptop, and so forth. Alternatively, the base may be any otherelectronic device that includes the HMD communication capabilitydescribed herein such as a smartphone, tablet, television, camera, andso forth. By yet other alternatives, the base 120, at a minimum, hasantenna and controls capable of beamforming and is able to store andtransmit images to the HMD along the beam whether those images are firstgenerated at the base or generated at another location, and stored onthe base. In some of these cases, the images and/or the beam positionmay be generated at some location remote from the base including aserver or computer whether communicating over a local area network orover a wide area network, camera, or other device, and transmitted tothe base by a wired or wireless connection. Many other examples can beused as well.

Referring to FIG. 2, an example image processing device or system 200 isshown for implementing the methods described herein and may be used tooperate system 100. The system 200 may include an HMD 202 with one ormore features similar to that of HMD 104, and a base 204 with one ormore features similar to that of base 120. In both cases, the HMD maytransmit sensor data to the base, and the base transmits image data tothe HMD with perspectives determined by using the received sensor data.The sensor data received at the base also may be used to determine thedirection of a radio-transmission beam to be directed to the HMD totransmit the images to the HMD. The HMD also may have the capability todirect a beam to the base to transmit the sensor data to the base and byusing its own sensor data. Sensor data used to direct the beams also maybe obtained from one or more sensors on the base and/or external to boththe HMD and the base. The beam position of the radio-transmission beamto transmit the sensor data from the HMD to the base may be determinedeither at the HMD or the base or other remote location. In the lattercases, the beam position or angle may be transmitted to the HMD for theHMD to set the beam or angle. By other forms, the base may have thecapability to set a receiving beam position to receive sensor data fromthe HMD rather than the HMD placing a beam at the calculated beamposition.

To perform these functions, the HMD 202 may have an on-board sensor unit206, which may include one or more sensors that at least sense theorientation of the HMD and provide data that indicate the orientation ofthe HMD. This may include orientation or head tracking sensors such asone or more accelerometers, gyroscopes, and/or inertial measurementunits (IMUs) such as magnetometers, HMD integrated camera and computervision systems for inside out positional tracking, external cameraarrays (one or more cameras) to track HMD position, structured lightemissions coupled with computer vision systems or scribe monkeys withnotepads taking concise notes on device positions, and so forth. Theorientation sensors either alone or in combination with other sensorscan determine the amount of change in the orientation from a referenceorientation such as an up-right forward facing zero orientation forexample and in three rotational degrees of freedom about the X, Y, and Zaxes shown on system 100 (FIG. 1) and to determine the tilt of the HMDin any direction. As explained below, a start orientation (which alsomay be referred to as an original, initialization, or referenceorientation) of the HMD is an actually measured orientation that may ormay not be in the zero orientation, or may be designated a [(0, 0, 0),(0, 0, 0)] orientation for the beam position determination operationsmentioned herein.

The on-board sensor unit 206 also may include sensors for positiontracking to determine the position and angular orientation of the HMD ina 3D space. In other words, this position measurement is a measure oftranslation along the x, y, and z axes of a 3D space formed by using thesensors as well as the angular orientation with respect to the referenceorientation. The HMD has an anchor point in the sensed 3D space formedby the sensor x, y, and z axes that is considered the location orposition point of the HMD. This anchor point is directly or indirectlyassociated with the antennas described below. Thus, the HMD and basepositions' anchor points may be antenna-based points that are a point onthe antenna or on the exterior surface of the body of the HMD and/orbase that is in proximity to the antenna or is some computed point thatis an average (or some other combination) of multiple branches of theantenna where the branches may be controlled individually or together tosteer a beam. Otherwise, the anchor point may simply be a middle or endpoint on the device or some other convenient point as long as the anchorpoint for position locating purposes is fixed on the HMD and base and issome fixed, known (or calculable) distance that is with respect to theposition of the antenna and that is considered the start and/or end (orsource and/or sink) point of the radio transmission beam, and for boththe HMD and the base.

Many different techniques for measuring the position of the HMD exist.Relative to the HMD, two general different types of position locatingtechniques are known, inside-out and outside-in. An inside-out techniquehas one or more position sensors (or receptors) on the HMD and thatmeasures the changes in distance to objects in the environment aroundthe HMD to determine its own position. An outside-in technique has thesensors exterior to the HMD and observes the movement of the HMD. By oneform, the sensors may be paired with a projector such as a laser orinfra-red (IR) emitter or other light projector. The projector projectslight out from the HMD for example, and the sensor on the HMD receiveslight reflected back from other objects in the room for the inside-outconfigurations. The outside-in techniques use the opposite configurationwhere the base 204 or some other external device may have the projectorand/or sensor and the light reflects off of the HMD. In some of thesecases, the objects to be detected may have markers with varyingsophistication in technology to act as reference points. Fiducialmarkers show in images only with a certain type of light. Some markersmay be simple reflectors, others may have barcodes or quick response(QR) codes that convey some sort of information, and so forth. Byanother approach, the receptor or sensor, whether on the HMD or thebase, receives light projected from the opposite device and is able todetermine its position from the direction of the light. In a lighthousetechnique, the projector floods a room with light for this purpose. Byyet other alternatives, sensors may be cameras such as a 3D or RGBDcameras that can reconstruct a 3D space and may provide a depth map. TheHMD described herein is not limited to any particular one orientationand position tracking technique, and many different such techniques, orcombinations of such techniques, may be used. Other sensor types thatcould be used include the sensors used for the orientation measurementsuch as the accelerometer and the IMU mentioned above, as well as aglobal positioning sensor (GPS), and so forth.

The on-board sensor unit 206 (or other sensors) may generate raw sensordata, and the raw sensor data that is used to create the perspective ofthe images to be displayed on the HMD is then compressed or otherwiseformatted by a sensor data compressor 208 for transmission to the base204. This may include packing of the data into a bitstream for exampleby known techniques. The compressed and/or packed sensor data then maybe provided to a (Tx) transmission unit 210 that uses an antenna 212 totransmit the sensor data along a radio-transmission beam to the base204. The Tx unit 210 may be considered separate from a (Rx) receiverunit 214 on the HMD or may be part of a transceiver that transmits andreceives data with the receiver Rx unit 210. The transmission andreceiver units 210 and 214 usually have antenna arrays of varyingconfigurations and antenna count that define the accuracy and width ofthe transmission beam generated by each device.

Also as mentioned, the network used to communicate between the antenna212 on the HMD and the antenna 240 on the base 204 should be a highbandwidth, low latency network such as WiGig by the example used herein,although other networks could be used. By one form, the network is ashort-range network in order to provide real-time video. 802.11ac Wave 2supports beam forming as well but may not have sufficient latency andthroughput characteristics to support AR/VR applications.

The antenna 212 may be one or more steerable beamforming antenna withbranches that can transmit in different phases to direct a beam in acertain direction when transmitting. The steering may be performedelectronically, mechanically, some combination of these or by otherways. The antenna 212 may be considered a single antenna or may includeat least two separate antennas with one antenna to receive the imagedata and one antenna to transmit the sensor data. The antenna 212 may becontrolled by a beam control unit 234 to steer the beam toward the baseas described in further detail below.

By another alternative, the antenna 212 cannot be steered and an antenna240 on the base can be steered for both transmitting and/or receiving.Thus, when a beam position is computed for transmitting at least sensordata from the HMD 202 to the base 204, the antenna 240 at the base 204may be controlled to direct a reception beam toward the antenna 212 forreceiving data. This may be a separate beam position than the beamposition used to transmit image data from the base to the HMD. Analternate implementation could be to employ an omnidirectional antennaon the HMD for sensor data using a low latency, low bandwidth networkingcapability.

During an initialization stage, the conventional methods may be used toperform a radio-based search to determine a beam position and steer aradio transmission beam to transmit the sensor data to the base.Thereafter, once a start HMD position relative to the base isestablished, an incident angle between an initial beam position and adesired current beam position may be determined and based on sensor datainstead of radio-based searches as described below. The incident angleis the beam position to transmit the sensor data from the HMD to thebase. The computations to determine the incident angle may be performedat the base or the HMD, and both alternatives are shown on system 200.

Turning for now to the operation of the base 204, the base antenna 240receives sensor data transmitted by the HMD 202. As with the antenna212, the antenna 240 also may include at least two antennas, at leastone for receiving sensor data and at least one for transmitting imagedata, or may be a single antenna that performs both tasks. By one form,at least the antenna transmitting image data may have multiple branchesor other structure to provide a beamforming antenna and in order tosteer the beam as mentioned with antenna 212.

A (Rx) receiver unit 242 may receive the sensor data, perform anydecompression and formatting that is required, and provide the sensordata to a sensor data analysis unit 244. The sensor data analysis unit244 also may receive sensor data from one or more external sensors 248including sensors which may or may not be on the base (where externalherein refers to external to the HMD), and/or external image capturedevices or cameras 246. The external sensors may be as described aboveto determine at least the position and/or the orientation of the HMD.The external cameras 246 may be used as a sensor as well as to captureimages of the HMD to determine the position and/or orientation of theHMD. Alternatively, such a camera may capture images of the surroundingarea near the user to form an environment in an augmented world, and/orto capture images of the user to construct an avatar of the user. Imagesof the user may be used to an avatar of the user in realistic poses in avirtual or augmented world viewed by more than one user each with theirown HMD 202 when desired. The sensors 248 and cameras 246 (andprojectors when used) may be communicatively connected, eitherwirelessly or wired, to the base or are considered mounted on, or partof, the base.

The sensor data provided from the HMD may be raw sensor data or may besensor data that has already been analyzed at the HMD so that a 3D HMDposition and/or orientation is provided to the base. Otherwise, thesensor data analysis unit 244 receives the raw sensor data from the HMDand determines the HMD position and/or orientation. In this case, thesensor data analysis unit 244 has an HMD orientation unit 250 thatanalyzes the raw sensor data and forms a current HMD orientationrelative to a zero orientation for example. This may includeestablishing an HMD orientation in rotational units (i_(n), j_(n),k_(n)) where i, j, and k are angles respectively about the HMD's own X,Y, Z, axes as shown on system 100, and where (0, 0, 0) is either thezero orientation or a designated start orientation. This is an HMDorientation established from the orientation sensors and alone is notyet relative to any other position of the HMD, and may or may not berelative to the position of the base depending on whether the sensorsalso determine the base position. Alternatively, these operations couldbe performed at the HMD as described below such that the HMD orientationunit 250 may receive the already determined HMD orientation form theHMD.

An HMD location unit 252 also may be provided to determine a current (x,y, z) HMD position of the HMD in an HMD or sensor 3D space, and by usingthe sensors mentioned above. Initially, this sensor-based HMD positionmay or may not be obtained relative to the base. Either way, the samecomputed HMD sensor orientation and sensor position can be used for bothdetermining image perspectives to display images as well as to determinea radio transmission beam position.

As to image generation, a video generation unit 254 then receives theHMD orientation and position data for a current HMD position. This mayinvolve determining the position and orientation of the HMD in a modelof the 3D space to generate the image to be displayed on the HMD. Ineffect, the position and orientation of the HMD indicates the positionof the display screen of the HMD in the 3D model space, and in turn, theposition and orientation of a virtual camera in the 3D space that wouldform such an image. For augmented reality, this may include the captureof real-world images from a camera and that are converted into a depthmodel and then projected to the depth model before the new image isformed. Once the image is formed, it may be placed in a video buffer 256until it is retrieved for compression and transmission to the HMD fordisplay.

After the images are formed for virtual or augmented reality, apre-processing unit 258 may apply pre-processing to the image datasufficient to perform encoding of the images for transmission to theHMD. However, when augmented reality is being performed and real-worldimages were captured by a camera, some of these tasks may be performedbefore or during the video generation to prepare the captured images for3D modeling as mentioned. Also, the type of pre-processing may depend onwhether the images are real world images obtained from a camera on theHMD for virtual reality or whether the images are entirely animatedimages from a fictional virtual world. The pre-processing unit 258 mayperform demosaicing, de-noising, filtering, color space conversions(such as RGB to YUV), resolution conversions, division into frames, andother pre-processing operations that may be needed for sufficient imageprocessing and compression desired as described herein. Otherpre-processing operations may include depth-sensing, depth-processing,and background/foreground segmentation (or keying) to name a fewexamples. Also, it will be appreciated that pre-processing units couldbe located on the HMDs and/or external cameras when used, in additionto, or instead of, the base 204, and may be provided in a differentorder than that shown on system 200 when needed.

A video encoder 260 then may compress the images based-on knownstandards such as HVEC, MPEG, VP#, and so forth before providing theimage data to a multiplexer 262 that adds other non-image data. This mayinclude computed HMD position or orientation data when such data is notcomputed by the HMD but is used by the HMD to transmit sensor data. Italso may include other image overhead and display control data fordisplaying the images on the HMD.

A (Tx) transmission unit 264 then formats the data for wirelesstransmission and sends the data out through antenna 240. A beam controlunit 274 may control the antenna 240 and steer a radio-basedtransmission beam toward the HMD and to a desired beam position asdescribed elsewhere herein. By one form, at least during aninitialization stage, the beam control unit 274 may use the antenna 240to perform a search for the HMD in order to aim the beam at the HMD andas described above. The image and other data is then received at theantenna 212 over the short range wireless network described above.

At the HMD 202, the antenna 212 may receive the image data from the base204. The receiver unit 214 on the HMD 202 receives the compressed imagedata and may provide the data to de-multiplexer 216 when combined withaudio data and/or other non-image data for controlling the HMD ordisplay parameters. A video decoder 218 then decompresses the imagedata, and a post-processing unit 220 performs post-processing tasks todisplay the images. This may include color space conversion (from YUVback to RGB, for example), resolution conversion, other scalingconversions, and so forth. The images then may be placed in a framebuffer 222 to be retrieved as needed by a video renderer 224 thatcontrols the images placed on a display 226 on the HMD 202.

Turning now to the beam position determination, and as mentioned above,an initialization stage is used to establish a more precise initial beamposition and a start HMD position and HMD orientation that issufficiently accurate to be used as a reference position and orientationduring run-time in order to determine the position of the current HMDposition relative to the base and a beam position to perform wireless,radio data transmissions. In the initialization stage, the base 204 mayestablish an initial radio-transmission beam directed to the HMD 202using known techniques, including the radio-based search for a strongestsignal or other indicator of the HMD location, and that initially can beused to transmit images to the HMD. Once the initial beam isestablished, the position of the HMD 202 relative to the base 204 may becomputed to set the start HMD position and start HMD orientation.Changes in position and orientation are then computed from the startposition and orientation going forward so that a current HMD positionrelative to the base can be determined each time, or individual times,the HMD is moved.

Many different ways may be used to track the HMD position relative tothe base. By one example, the HMD location unit 252 may determine thestart position of the HMD relative to the base during the initializationstage by using two or more of the obtained sensor-based HMD positions.Specifically, and to provide the beam positions during initialization,the user may hold the HMD in the start position, and then may move to atleast one more position wearing or holding the HMD in a fixed non-movingstate at each position. The instructions to the user to move from oneposition to the other may be by audio or visual instructions to the usersuch as by instructions shown on the HMD display 226 to stay at alocation while wearing the HMD for a specific time period at eachlocation. By one example, the user wearing the HMD is to move directlysideways in front of the base so that the only significant change fromposition to position is in one direction (x) while the distance (y) awayfrom the base (shown as distance d (FIG. 5)) and height z distancesremain at least generally the same. The distance d is the perpendicularlength from the base to the line connecting two HMD initializationpositions as explained below. The y (or d) and z distances are assumedto remain constant for the start HMD position calculation example here.

A radio-based search is then performed at a number of initializationstage HMD locations to set the beam position for each of thoselocations. The radio-based search may include a search for the strongestradio signal while the user holds or wears the HMD at the HMD position.The strongest signal found by the search should be the antenna 212 ofthe HMD in most cases. The base 204 receives the data of the signalstrength in a certain relatively wide area near the base. Once the HMDis found, and specifically the location of the antenna 212 on the HMD,the beam is directed in that direction toward the HMD.

Once directed, the angle of the beam position is now known by the base.Thus, the beam control unit 274 then may provide indication of aninitial beam position directed toward the initial or start HMD positionand at least one more beam position directed toward other HMDinitialization positions. The indication or angle of the beam positionmay be provided to an HMD initializing unit 268 of a beam directiongeneration unit 266. The angles of the beam positions are designated asthe α′ and β′ angles described below (FIG. 5) for this example. Theseangles indicate the beam position direction relative to a base referenceline (here along line or distance d that extends perpendicular from thebase).

The HMD initializing unit 268 also retrieves the sensed (x, y, z)position coordinates of the HMD initialization positions from the HMDlocation unit 252 and once the beam position is established for thesensed position. The length L is the length between two HMD positionsmeasured by subtracting the sensed position coordinates

In more detail, the distance d extends perpendicularly from both thebase and the length L. Particularly, the distance d extendsperpendicularly from a base front or reference line Ba that includes thepoint where radio signals are received or transmitted and the radiotransmission beam starts or is received, such as an antenna-relatedpoint as mentioned above. Once the initializing unit 268 determines thedistance angles α′ and β′ and the length L between the two HMDinitialization positions, then the distance d as well as the length fromthe perpendicular distance to the individual start HMD position can becalculated. The result is an (d) distance from the base, which may belabeled at position (0, 0) and to the start HMD position, which is alsothe initial beam position running along an HB (FIG. 6) beam line ordistance from the start HMD position to the base. The (d) beam position(or start HMD position or HMD_Position_One) then may be saved todetermine a change in HMD position relative to the start HMD positionand during run-time use of the HMD. The (d) directions become the [x, y]directions in a 2D or 3D space defined by the start HMD position and thebase. While this calculation assumes the user wears the HMD at the sameheight when moving from HMD position to position during theinitialization stage so that there is negligible change in height, itwill be understood that the same calculations can be made by adding achange in height z also as described below.

As to the initial orientation of the HMD in the start HMD position, theHMD orientation unit 250 also may provide the sensed orientation in (i1,j₁, k₁) degrees in rotation about the HMD X, Y, Z axes and relative tothe zero orientation which is an upright-straight forward orientation asdescribed above. This orientation of the HMD in the start HMD positionalso may be stored to be differenced with subsequently obtained currentHMD orientations to compute a change in orientation during run-time.This change in orientation then can be used to determine a beam positionto transmit sensor data from the HMD back to the base as describedbelow.

During run-time, a new beam position may be determined each time the HMDhas moved some minimum amount (whether in position or orientation) ormay be updated after a certain time interval regardless of motion. Thismay or may not be the same as that used for updating the imageperspectives. Either way, the sensors provide new sensor data from theHMD, base, or other location, and to the base to update the sensorposition of the current HMD position, and is provided to the HMDlocation unit 252 as described above. The HMD location unit 252 providesthe current sensed HMD position in (x, y, z) coordinates of the 3D spaceof the HMD that is the same sensed 3D space as the sensed coordinates ofthe start HMD position. This current sensed HMD position is provided tothe base-to-HMD beam unit 270.

Also, the HMD orientation unit 250, as mentioned above, generates thevalues for the current angular orientation of the HMD about its own X,Y, Z axes such as (i_(n), j_(n), k_(n)) that are the rotational degreesabout the respective axes and relative to the zero orientation. This isan HMD orientation established from the orientation sensors and is notyet relative to any other position of the HMD, and may or may not berelative to the position of the base depending on whether the zeroorientation of the HMD was set parallel to the x, y, z, axes forming the3D space of the base 204 for example. The HMD orientation unit 250provides the current orientation of the HMD to the current HMD-to-basebeam unit 272.

The base-to-HMD beam unit 270 determines the change in HMD position fromthe stored start HMD position to the current HMD position by determiningthe difference in each of the three (x, y, z) coordinates between thetwo positions. The change is then added to the (d) base-relativecoordinates of the start HMD position thereby obtaining adjustedcoordinates of the current HMD position but now relative to the base.The coordinates of the current HMD position can then be used todetermine an updated radio transmission beam angle α′ for an updatedbeam position. This beam position may be used for transmitting imagedata from the base to the HMD and is provided to the beam control unit274 to control the antenna 240 to direct the beam along the new beamposition and to the current HMD position. These beam positions aredetermined without run-time radio-based searches to set the beampositions.

As the beam position to transmit sensor data to the base, the currentHMD-to-base beam unit 272 determines an incident angle on which theradio beam is to be positioned to transmit the sensor data (and otherdata when desired) from the HMD and back to the base. The incident anglefactors both (1) the change in angle from the initial beam position formthe base to the start HMD position and to a desired beam positionextending from the base to the current HMD position, as well as (2) thechange in orientation angle from the orientation of the HMD in the startposition to the orientation of the HMD at the current position.

To determine the first (position) component of the incident angle, thecurrent HMD-to-base unit 272 uses the sensed coordinates of the startand current HMD positions to determine a change in each of the three (x,y, z) coordinates. The current HMD-to-base unit 272 then determines thechange in angle from the start HMD position to the current HMD position,and relative to the base. This is accomplished by using the initial beamposition H as an axis (such as an x axis) where the y and z axes areperpendicular to the beam position H. Then, the change in angle relativeto the initial beam position and the base is determined by usingtrigonometric equations, which results in position angles (Φ_(xn),Φ_(yn), Φ_(zn)) defining a line (or vector P as in FIG. 6) from thecurrent HMD position to the base, and from initial beam position H. Thedetails are provided below.

To determine the second (orientation) component of the incident angle,the current HMD-to-base beam unit 272 determines the difference betweenthe orientation or rotational position of the HMD at the start andcurrent HMD positions. These differences are added to the positionangles to determine a final incident angle Φ_(HMD). The incident anglethen may be transmitted back to the HMD when the incident angle iscomputed at the base or other location rather than the HMD, andimplemented by steering the antenna 212 at the HMD 202. Otherwise, theincident angle may be provided to the beam control unit 274 to controlthe antenna 240 at the base 204 to form a beam at a receiving beamposition to receive sensor data from the HMD, either way being performedwithout the use of a radio-base search.

Turning to run-time beam position operations on the HMD, by one form,the HMD 202 receives the incident angle as calculated by the base 204.By this approach, the incident angle is transmitted to the antenna 212of the HMD, and is extracted and provided to the beam control unit 234.The beam direction control unit 234 then directs the beam from the HMDto the base 204 to transmit sensor data to the base.

By an alternative form, the HMD 202 computes the incident angle insteadof, or in addition to, the base 204 performing the incident anglecomputation. In this case, the sensor data from the on-board sensor 206,base 204, or other sensing devices may be provided to a beam directiongeneration unit 228. It will be understood that the provided sensor dataafter the initialization stage does not include or primarily rely on aradio-based search, such as a signal strength search by one example,performed to steer a radio transmission beam. The beam directiongeneration unit 258 may have a sensor data analysis unit 230 and a beamdirection calculation unit 232. The sensor data analysis unit 230 mayreceive raw sensor data and analyzes the data to at least determine acurrent orientation of the HMD, as described with the HMD orientationunit 250. The sensor data analysis unit 230 may also receive and/orgenerate sensor position (x, y, z) coordinates of the current HMDpositions, but also could be used to compute the coordinates of thestart HMD position during the initialization stage for example.

Thereafter, the beam direction calculation unit 232 may use the currentHMD orientation and position to determine the incident angle of the beamposition or direction to transmit the sensor data to the base 204.Specifically, the start position and orientation may be obtained from amemory on the HMD or as transmitted from the base. The beam directioncalculation unit 232 also computes or otherwise obtains the distance Hfrom the base to the start HMD position. The change in orientation (i,j, k) and position (x, y, z) between the start and current orientationand position respectively then may be computed. Once these changes arecomputed, a single total incident (or final beam) angle from the currentHMD orientation and position to the base may be determined by the beamdirection calculation unit 232 using the same algorithms as explainedfor the current HMD-to-base unit 270 except that these computations areperformed at the HMD for this example. The incident angle then may beprovided to the beam control unit 234, which in turn controls the phaseand/or other factors of the antenna 212 to transmit sensor data alongthe incident angle to the base, thereby establishing the beam at thedesired beam position without the use of a radio-based search.

By another alternative, once the base 204 establishes an initial beamposition, thereafter, the HMD 202 may generate the beam positions fortransmission both to and from the base 204. By another option, he HMDmay perform all beam position calculations, including during theinitialization stage, where the HMD determines beam positions andtransmits the beam positions to the base when the base is to set eithertransmitting or receiving beam positions. In any of these cases, the HMD202 performs the computations and transmits the desired beam position tothe base 204 to steer the beam in the correct direction to transmitimages to the HMD 202. Otherwise, the HMD may provide the beam positionangle to its own beam control unit 234 to steer a receiving beam atantenna 212 to receive image data from the base, either in addition tothe steering at the base, or instead of the steering at the base (wherethe base does not have steering ability).

It will be appreciated that while a single HMD is shown, multiple HMDscould be controlled to provide a multi-user environment and by using aseparate antenna system for each HMD whether by a single base ormultiple bases. Other systems could use a shared antenna for multipleHMDs. One approach to implementing a multi-user environment is to assigna 1:1 relationship between each HMD and a corresponding base within thesame environment. Ideally, all of the bases would be distributed broadlyacross the environment to reduce the likelihood of an HMD not pairedwith a given base transmitting data to that base. This configurationwould work best with bases installed on the ceiling of the environmentto minimize obstruction of the beam by the users in the environment.

It will be appreciated that other components not shown may be providedfor the system 200, such as those shown with systems 1000, 1100, and/or1200 described below. It also will be appreciated that a depictedcomponent includes code and/or hardware to perform the function of thedepicted component and may actually be located in a number of differentplaces or components on a device that collectively perform the recitedoperations of the depicted component.

Referring now to FIG. 3, by one approach an example process 300 is acomputer-implemented method of wireless data transmission for virtual oraugmented reality head mounted displays. In the illustratedimplementation, process 300 may include one or more operations,functions or actions as illustrated by one or more of operations 302 to310 numbered evenly. By way of non-limiting example, process 300 may bedescribed herein with reference to example image processing systems 100,200, 500, 600, 1000, 1100, or 1200 of FIGS. 1-2, 5-6, and 10-12respectively, and where relevant.

Process 300 may include “sensing a 3D position of at least one headmounted display (HMD) arranged to display video sequences to a person”302. As mentioned herein, a user wearing the HMD with one or moredisplays is shown images on display screens so that the user views avirtual or augmented reality which may be in point of view (POV) so itseems that the user is in the virtual or augmented reality. Sensors onthe HMD or external to the HMD, such as on a base communicating with theHMD or some other external location, may be used to detect a 3D locationand orientation of the HMD in a 3D space, and this may occur after eachor individual times the HMD is moved, or after certain time intervals.

By one example, an initialization stage is performed, and the beamposition is initially determined using radio-based search methods for atleast two HMD locations where the HMDs are held still some time periodfor the initialization stage. The angle of the beams from the base tothese positions are then known. The sensed HMD locations and the angleson these conventional beams are then used in a triangulation algorithm,by this example, to determine a start 3D position that is relative tothe base. Both the start HMD position and orientation of the HMD at thestart HMD position may be saved for later use.

Thereafter the initial stage, however, whenever the HMD is moved, orafter some time interval, the beam position may be updated by using thelatest sensed position, and orientation, of the HMD without using atime-consuming radio-based search. The following operations are directedto those operations after the initialization stage.

Thus, process 300 may include “determining the location of the sensed 3DHMD position relative to a base” 304. In other words, the sensed 3D HMDposition may be in a 3D space that is typically used to determine theposition of the HMD relative to other positions of the HMD, and thelocation within the 3D space itself. The position of the base isnormally not needed to determine such 3D positions. Here, however, theposition of the HMD relative to the base is used to determine beampositions.

By one approach then, the change in position from the start HMD positionto a current HMD position is determined, and in one example, bydifferencing the coordinates of the two positions and adding thedifference to the start HMD position, resulting in a current HMDposition that is relative to the base.

Thus, process 300 may include “wherein the sensing and determining isperformed with other than a radio-based search” 306. Specifically, afterthe initialization stage, even though radio-based transmissions at abeam position are to be used to transmit data between the base and theHMD, the location of the beam position is not determined by using aradio-based search. Other sensors as described elsewhere herein are usedinstead.

Process 300 may include “determine a beam position of a wirelessradio-based transmission beam by using the 3D HMD position” 308. Thisposition, now with coordinates relative to the base, can then be used tocompute the angle of a beam position from the base to transmit databetween the base and the HMD.

Process 300 then may include “wirelessly transmit images from the baseand directed toward the HMD along the beam to display the images to theperson and at the HMD” 310, and particularly, where the data istransmitted along the beam position established by using updated senseddata rather than a radio-based search.

Many variations and alternatives to the process provided above exist.The HMD 3D positions may be used to determine a second beam position totransit sensor data from the HMD and to the base by determining anincident angle from the HMD (or more particularly, the initial beamposition). This is performed by factoring both (1) the change in anglerelative to the base and associated with the change in HMD position andchange in beam position, and (2) the change in orientation of the HMD atthe start position to the orientation of the HMD in the current HMDposition.

The computations for determining either of the beam positions (fortransmitting images or sensor data) could be at the base, the HMD, orboth, and in any combination such as both on one device (the HMD orbase) or one each at one device. Also as mentioned, there may bemultiple HMDs each with its own base or antenna system or the base andantennas may be shared.

By another alternative, the change in position and orientation may bethe change from the last HMD position in a series of HMD positions,rather than the start HMD positions. In this case, the change in HMDposition and orientation is simply added to that of the last positionanalyzed to provide a position relative to the base.

Referring now to FIG. 4, by one approach, an example process 400 is acomputer-implemented method of wireless data transmission for virtual oraugmented reality head mounted displays. In the illustratedimplementation, process 400 may include one or more operations,functions or actions as illustrated by one or more of operations 402 to422 numbered evenly. By way of non-limiting example, process 400 may bedescribed herein with reference to example image processing systems 100,200, 500, 600, 1000, 1100, or 1200 of FIGS. 1-2, 5-6, and 10-12respectively, and where relevant.

Process 400 includes one example initialization process to establish therelative positions of the HMD and the base with sufficient accuracy toprovide a start or initial HMD position in 3D space that includes aposition of the base. Subsequent HMD positions relative to the base canthen be determined by measuring the change in HMD position directly orindirectly form the start HMD position so that the HMD image processingsystem can reduce or eliminate radio-based searches for the HMD in orderto direct a radio-based transmission beam to or from the HMD, at leastafter the initialization process. By this approach, the base is at afixed location and is described as above. At a minimum, the base is tobe fixed once initialization commences and until the start HMD positionis refreshed or updated.

Preliminarily, one or more users each with their own HMD may initializethe HMD to determine an HMD position relative to the base. The user maybe provided instructions on the display of the HMD, electronically onthe device operating the HMD image processing system (or HMD system forshort), other devices, or by other ways. The user is to wear the HMD andstand near the base, and in the base's field of operation when limited.The user is to stand as still as possible while a first or start HMDposition is determined, and when indicated, or after a specified timeperiod, the user is to move sideways in front of the base to at leastone more second (or subsequent) position and then stand still again.

Referring to FIG. 5, this arrangement is displayed on an HMD system 500that has an HMD 502 placed at a start HMD position 504 and then a secondHMD position 506 both in front of a base 508. Other details will bedescribed below. The user may move at least once but could be more timesto form some sort of average position value. For the exampleinitialization process described herein, it is assumed that the userwearing the HMD and moving sideways in front of the base while standingdoes not change the height or z coordinate in 3D space of the HMD. It isalso assumed that by the user moving sideways in front of the base theperpendicular distance d (on system 500), or x coordinate of the 3Dspace by this example, also remains constant so that the movement of theHMD 502 from the start HMD position to the second HMD position alongpath L for example is parallel to the base antenna reference line Ba atthe base so that distance d is perpendicular to both L and the baseforward reference line Ba.

The line Ba on the base 508 includes the point that is considered thetransmission point or start of the radio-based transmission beam, andmay be the same point for receiving data from the HMD although it couldbe different. Likewise the points H1 (504) and H2 (506) are alsoconsidered the radio beam transmission and/or reception points but couldbe different points. As described above, these points may or may notrelate to the structure and position of the antenna on each of thedevices. Thus, the 3D (x, y, z) coordinates for the HMD and the base maybe start and end points of the beams but could be other locations onthose devices that are to be considered the 3D space anchor points forthose devices. Thus, while the end point of a beam may be different thenthe actual 3D location of one of the devices, it will be understood thatthe distance from one to the other, when not the same, can be fixed forthe device and determined easily when performing computations. Forclarity, it may be assumed when convenient herein that the beamendpoints are located at the same 3D anchor positions as the HMD andbase.

Process 400 may include “establish initial beam between HMD andnon-moving base with HMD in at least a generally fixed position” 402. Insome forms, the 3D space established by the sensors will not determinethe position of the base relative to the HMD at all or with sufficientaccuracy to transmit data between the base and the HMD. In these cases,the radio-based beam search may be performed during initialization. Thismay include determining the strongest radio signal by a number of knownradio-based search operations, and then controlling antenna brancheseither mechanically and/or electronically to perform beamforming andsteer a transmission beam toward the HMD. When the beam is established,the angle of the beam position is known to the beam control unit orbeamforming control, here referred to as angle α′ (FIG. 5), and wherethe angle is relative to the Ba reference line, or here the distance dsince the distance d is assumed to be perpendicular to the Ba referenceline. Once the beam position angle is established for the start HMDposition, the base may obtain the sensed position data of the positionof the HMD where it receives the beam transmission.

Accordingly, process 400 also may include “sense initial HMD location”404 also referred to as the start HMD position. As already described indetail above, many different types of sensors may be used whethermounted on or in the HMD, base, or other external location. Asmentioned, this may involve optical, infra-red, or other beam detection,whether direct from a projector or from reflectors on the HMD or base.The sensors also may include cameras that form depth maps or otherwisegenerate a 3D space and locates the position of the HMD within the 3Dspace. A GPS-type of sensor also may be used when sufficiently accurate.Also, this operation may include the use of internal sensors on the HMDthat may or may not also be used to determine the orientation of the HMDas well, such as head tracking sensors including one or moreaccelerometers, gyroscopes, and/or inertial measurement units (IMUs),and so forth. Many different sensors and combinations of sensors can beused. The result is a sensed (x, y, z) position coordinate of the startHMD position.

Once the sensed start HMD position is established, the process 400 thenmay include “correlate HMD location to initial beam position” 406, or inother words, determine the start HMD position relative to the baseposition. By this example, process 400 may include “move HMD at leastonce to a new location” 408 and to a second 3D HMD position as a user iswearing the HMD as mentioned above or otherwise moved in a differentway, such as held by the user.

Process 400 may include “establish beam at new location(s)” 410, and asdescribed above to perform a radio-based search to determine a strongestsignal or other indicator of the HMD position, and then to direct aradio-based transmission beam at the HMD in the second HMD position (H2in FIG. 5). This operation also includes determining the beam positionangle (3′ (FIG. 5) as the beam is directed from the base and to thesecond HMD position, and as mentioned above for a′.

Process 400 may include “determine position of HMD(s) at newlocation(s)” 412, and again by using sensors to determine the (x, y, z)coordinates of the HMD at the second HMD position and in the same 3Dspace as the start HMD position.

While still referring to system 500 (FIG. 5), process 400 may include“determine start HMD position relative to the location of the base usingtriangulation with direction of beams and HMD locations” 414. Bytriangulation, the distance d to the base as well as the distance

from the vector {circumflex over (d)} (or distance d) to the start HMDposition H1 may be determined by using the included angles {circumflexover (α)} and {circumflex over (β)} which are the angles between thelength L connecting the two HMD positions and the established beampositions HB1 and HB2 as follows:

Angle α′ is known and indicates a specific beam position formed when theHMD is located at the start HMD position, and the included angle α is:

α=90−α′  (1)

Angle β′ is known and indicates a specific beam position formed when theHMD is located at the second HMD position, and the included angle β is:

β=90−β′  (2)

Length L between the two discrete HMD positions H1 and H2 is calculatedas:

L=|√{square root over (((x ₂ −x ₁)²+(y ₂ −y ₁)²+(z ₂ −z ₁)²))}|  (3)

where the start HMD position is (x₁, y₁, z₁) while the second HMDposition is (x₂, y₂, z₂). Applying a triangulation algorithm, distance dis:

$\begin{matrix}{d = \left( \frac{L*{\sin (\alpha)}{\sin (\beta)}}{\sin \left( {\alpha + \beta} \right)} \right)} & (4)\end{matrix}$

With d and α′ known, length |

|, as shown on system 500, describing the distance from the start HMDposition to the vector {circumflex over (d)} (at position d) that isboth perpendicular to

and intercepts the base beam transmission source point.

|

|=tan(α′)*d  (5)

The position of the HMD with respect to the base is now established atthe start HMD position where:

HMD_Position_One≡[|

|,d]  (6)

where the base transmission source point may be set as (0, 0, 0) and [|

|, d] can be defined as [x, y] coordinates from the base, and goingforward can be used to determine any HMD position relative to the startHMD position, which in turn, is relative to the base. Further detailsare provided below with the run-time process 700.

For this example, it is assumed z=0 so that z is eliminated for thisequation, but it will be understood that the same above equationsadjusted to include a z dimension can be used when there is a change inheight z from the start HMD position to the second or subsequentpositions. By one approach, this same process may be applied todetermine beam transmission angles along vertical HMD transitions. Thisprocess would use an initial position followed by a subsequent HMDposition displaced along the z-axis.

Process 400 may include “determine orientation of the HMD at the startHMD position” 416. The orientation is determined by the head trackingsensors mentioned above or other sensors that can provide such anorientation, and which may be on or internal to the HMD but may be atthe base or other external location depending on the type of sensor. Theangle of the HMD relative to its own X, Y, Z axes and zero orientationas described with system 100. This may provide an orientation of the HMDat the start HMD position and facing in (i₁, j₁, k₁) degrees each beingan angle about a respective X, Y, and Z axes.

Process 400 may include “store the start HMD position relative to thebase and the start HMD orientation” 418, where at least the start HMDposition (d) relative to the base is stored but the sensed (x, y, z)start position may be stored as well, and the orientation (i, j, k) ofthe HMD at the start HMD position is stored.

Then during run-time, process 400 may include “determine a change inposition from the start or prior HMD position to a new HMD position whenthe HMD is moved” 420. This operation is detailed below with process 700where the change in position from the start HMD position to a currentposition is determined, and then process 400 may include “determine anew beam position by using the change in HMD position” 422. This isperformed without performing a time consuming radio-based search. It isalso understood that instead of the start HMD position, the currentposition may be compared to any prior HMD position that already has aposition established relative to the base. The details and othervariations are provided below.

By an alternative approach, the radio-based search may be eliminatedduring the initialization stage as well when the sensors are able todetermine the location of the multiple HMD positions and the baseposition each of which indicate a start and end position of transmissionbeams for the devices (when the position point is not the same as thebeam endpoints) and with sufficient accuracy to be able to rely on thosepositions for transmission of data along the beams.

Referring now to FIG. 7, by one approach an example process 700 is acomputer-implemented method of wireless data transmission for virtual oraugmented reality head mounted displays. In the illustratedimplementation, process 700 may include one or more operations,functions or actions as illustrated by one or more of operations 702 to718 numbered evenly. By way of non-limiting example, process 700 may bedescribed herein with reference to example image processing systems 100,200, 500, 600, 1000, 1100, or 1200 of FIGS. 1-2, 5-6, and 10-12respectively, and where relevant.

Now during run-time, the user may wear the HMD and move his head whichwill change the perspectives and position of the image that should bedisplayed to the user in real-time. The description of the HMD and baseis as described above. To provide a radio transmission beam directedfrom the base and to the HMD to efficiently transmit images from thebase to the HMD along the beam position, process 700 may include “senseposition of head mounted display (HMD)” 702. This includes sensing acurrent HMD position in the 3D space of the HMDs and determiningcoordinates (x_(n), y_(n), z_(n)) by using the position sensors asdescribed above.

Optionally, process 700 may include “transmit sensor data to base” 704.Thus, in these arrangements, the HMD may transmit the raw sensor data tothe base or other device to compute the current sensed HMD position atthe base or other device, or the sensed HMD position (x_(n), y_(n),z_(n)) may be computed at the HMD, and the coordinates are transmittedto the base or other device. In either case, the base or other device isthen to use the sensed current HMD position to compute a change inposition from the start or prior HMD position relative to the base. Inalternative cases, some part or all of the computations are performed atthe HMD, and the HMD transmits a computed change in HMD position, orfinal beam position angle, to the base or other image transmittingdevice for use. This may occur whether or not the HMD transmits thesensor data to the base or other device to generate images to bedisplayed on the HMD.

Process 700 may include “obtain start or prior HMD position” 706, andparticularly, the start or prior position is obtained by whicheverdevice, HMD or base (or other device) that is to determine thedifference between the start or prior HMD position and the current HMDposition. Also, while the example described herein mainly refers to thearrangement where each or individual current HMD positions aredifferenced from the designated start HMD position formed duringinitialization, it will be understood that alternatively, once a currentHMD position is established and that is relative to the position of thebase and start HMD position, that current HMD position can become aprior HMD position used as a reference for a new current HMD positionwhen the HMD is moved again. Since the position of the prior HMDposition is based on the start HMD position, it should have a positionsufficiently reliable to be the basis for the next new current HMDposition. This may result in a chain or series of HMD positions whereeach current HMD position relies on the immediately prior, or some otherchosen prior, HMD position rather than relying on the start HMD positiondirectly. Such a series, however, may not be as accurate as relying onthe start HMD position directly since there are more chances for sensorposition and other errors to enter the equations with the series.

Process 700 may include “compute current HMD position relative to base”708. This may include “use change in HMD position from prior or startHMD position” 710. The generic current HMD position can be defined as

Cur_HMD_Pos[x,y,z]  (7)

where in the current example for simplification, z=0 when a user wearingan HMD does not change the height of the HMD while the user is moving.Any arbitrary HMD position with respect to the base can now becalculated based on the high precision position data established duringthe initialization process (or carried forward to a prior HMD position).Assuming the start HMD position is being used here for this example, ateach instance the change in position measured in the HMD system can beapplied

BASE_HMD_Pos[x]=HMD_Position_One[x]+ΔHMD_POS[x]  (8)

BASE_HMD_Pos[y]=HMD_Position_One[y]+ΔHMD_POS[y]  (9)

where the start HMD position (or HMD_Position_One) [x, y] are the [

, d] values established during initialization (or values of a prior HMDposition in a series based on those initialization values). Thus, theΔHMD_POS[x] and ΔHMD_POS[y] is the difference between the sensedcoordinate values (x₁−x_(n)) and (y₁−y_(n)) respectively. To account forchanges in height of the HMD the same process may be applied except witha factor for the z values.

Once the current HMD position relative to the base is known(BASE_HMD_Pos[x] and BASE_HMD_Pos[y], or more generally labeledBASE_HMD_Pos[x, y, z], then process 700 may include “compute current HMDbeam position” 712. This may involve computing an angle describing thedirect vector from the base to the current HMD position and may becalculated at any instant as:

$\begin{matrix}{\alpha^{’} = {\tan - {1\left( \frac{{BASE\_ HMD}{{\_ Pos}\lbrack x\rbrack}}{{BASE\_ HMD}{{\_ Pos}\lbrack y\rbrack}} \right)}}} & (10)\end{matrix}$

where the beam position a′ angle is that as already described and asshown on FIG. 5, except now being an angle between the distance d lineand the current HMD position.

Process 700 may include “direct beam in direction of HMD” 714, where thebeamforming or beam control unit then aims the beam along the beamposition indicated by the angle α′ from the base. The process 700 thenmay “transmit image data along beam to HMD” 716, and so that he HMD canthen “display image(s) on HMD” 718, and as already described with HMDsystem 200. As mentioned, the transmission may be performed on ashort-range radio-based wireless network with high bandwidth and lowlatency such as WiGig although other network types as mentioned abovemay be used as well.

Also as mentioned, this is one example algorithm for determining thecurrent HMD position relative to the base and without performingradio-based searches at least during the run-time of the HMD, althoughit may be used during an initialization stage as described above.

Referring to FIG. 8, by one approach an example process 800 is acomputer-implemented method of wireless data transmission for virtual oraugmented reality head mounted displays. In the illustratedimplementation, process 800 may include one or more operations,functions or actions as illustrated by one or more of operations 802 to818 numbered evenly. By way of non-limiting example, process 800 may bedescribed herein with reference to example image processing systems 100,200, 500, 600, 1000, 1100, or 1200 of FIGS. 1-2, 5-6, and 10-12respectively, and where relevant.

Process 800 may include “sense position of head mounted display (HMD)using HMD sensors” 802, and as described above determining thecoordinates (x_(n), y_(n), z_(n)) of the current sensed HMD position inthe 3D HMD space that may or may not yet be relative to the position ofthe base.

Process 800 may include “sense orientation of HMD” 804. Here, the sensedorientation of the HMD is obtained in the three degrees of freedom(i_(n), j_(n), k_(n)) respectively about the X, Y, and Z axes of the HMDas described above. As mentioned, the head tracking sensors aretypically on the HMD but may be at least partly on the base or otherlocation external to the HMD depending on the type of sensor that isbeing used.

Process 800 may include “compute current HMD position relative to base”806, and as provided by process 700 to establish the spatialrelationship between the HMD and the base, and that may be used toprovide a beam position to transmit image data from the base to the HMDto display the images. The result is (x, y, z) or (d) coordinates of thecurrent HMD position that are relative to the (0, 0, 0) position at thebase.

Process 800 may include “determine incident angle of beam position fromHMD to base” 808. Specifically, once the spatial relationship betweenthe HMD and the base is established, the system may determine theincident angle of the transmission beam position extending from the baseand to the current HMD position. As discussed above, this may refer toprecisely extending between antennas or other anchor positions of thebase and HMD. This incident angle Φ_(HMD) also defines the beamtransmission angle required by the HMD to send information back to thebase. Two angular components exist to determine the incident angle, (1)the change in angle due to the change in the spatial relationshipbetween the base and the HMD, and (2) the change in the HMD orientationangle at the current HMD position compared to the orientation of the HMDat the start HMD position. The example computation of each of these twoincident angle components are explained in turn.

The first component of the incident angle is the angle formed betweenthe initial beam position (or line) extending from the base and the HMDon one side, and the line extending from the base and the current HMDposition assuming no HMD rotation from the start HMD Position to thecurrent HMD position. Given coordinates from the HMD system, the systemcan apply known trigonometric relationships to determine this value.Thus, process 800 may include “determine change in HMD position-relatedangle” 810. By one example and as described above, the distance Hbetween the HMD and base at the start HMD position, and along theestablished initial beam position a′ is:

H=|HMD_Position_One|=|[|l|,d]|

as described above when assuming no change in height z, or could bedesignated |[l|, d, z]| when constant height is not assumed.

Referring to FIG. 6 for this example, a separate coordinate system iscreated to determine the position-related component of the incidentangle, and system 600 is provided to exemplify this separate coordinatesystem. The HMD 602 is shown at the start HMD position H1 (604) and maybe moved to another or current HMD position Hn (606) during run-time andin front of a base 608 as described above. The start HMD position alsois generated during an initialization stage also as described above.

Here, a separate coordinate system is established with the x-axisrunning down the initial beam position H. With the initial beam positionH set as the x-axis, the y and z-axes extend in perpendicular directionsfrom the initial beam position (x-axis) H as shown. Since the start HMDposition is on the H (the x-axis), y=0 and z was assumed to be 0 aswell. Thus, the initial point (or start HMD point) may be written as:

HMD_Anglel_Pos≡[H,0,0]  (12)

The difference between the (x, y, z) coordinates of the start HMDposition and the current HMD position, and being those coordinatesrelative to the base, can be differenced to determine changes in each ofthe x, y, and z directions as ΔHMD_POS[x], ΔHMD_POS[y], and ΔHMD_POS[z],and the HMD Position Angle at any instance is defined as:

HMD_Anglen_Pos≡[H+ΔVR_POS[x],ΔVR_POS[y,ΔVR_POS[z]]  (13)

Once the current HMD Position with respect to the base is known theangular position sub-components are calculated as:

$\begin{matrix}{\Phi_{zn} = {\tan^{- 1}\left( \frac{H + {\Delta \; {{HMD\_ POS}\lbrack x\rbrack}}}{{\Delta HMD\_ POS}\lbrack y\rbrack} \right)}} & (14) \\{\Phi_{xn} = {\tan^{- 1}\left( \frac{\Delta \; {{HMD\_ POS}\lbrack y\rbrack}}{{\Delta HMD\_ POS}\lbrack z\rbrack} \right)}} & (15) \\{\Phi_{yn} = {\tan^{- 1}\left( \frac{H + {\Delta \; {{HMD\_ POS}\lbrack x\rbrack}}}{{\Delta HMD\_ POS}\lbrack z\rbrack} \right)}} & (16)\end{matrix}$

where each angle represents a sub-component of the position component Pthat is part of the incident angle. There are a number of ways todescribe these sub-components. One way is to state that eachsub-component Φ_(xn), Φ_(yn), and Φ_(zn) extends between an axis and theprojected vector in one of the planes formed by the axes, and its angleis relative to rotation about the third axis. For example, Φ_(zn) refersto the sub-component vector xy of the position component P and has itsangle measured around the z axis, and extends in an xy plane as shown onsystem 600. Likewise, Φ_(xn) has a sub-component vector yz that extendsin the yz plane and has its angle measured about the x axis, whileΦ_(yn) has a sub-component vector xz that extends in an xz plane and hasits angle measured about the y axis as shown on system 600. Thesub-component angles Φ_(xn), Φ_(yn), and Φ_(zn) can then be input to atotal incident angle equation (18) described below.

Turning to the orientation component of the incident angle, process 800then may include “determine change in HMD orientation” 812, which isperformed to determine the relative rotation of the HMD Φ_(Φn) about itsown XYZ axes and change in orientation from the start HMD positionorientation (i₁, j₁, k₁) to the current HMD position orientation (i_(n),j_(n), k_(n)), as shown on system 600, and is simply the change inrotation about each axis as measure by the HMD system:

Φ_(Φn)=[ΔHMD_Angle[i],ΔHMD_Angle[j],ΔHMD_Angle[k]]  (17)

where ΔHMD_Angle[i]=i₁−l_(n), ΔHMD_Angle[j]=j₁−j_(n), andΔHMD_Angle[k]=k₁−k_(n).

Process 800 may include “compute total incident angle” 814. The incidentangle or net angular displacement of the HMD which defines the requiredangle for the HMD radio or transmitter to transmit data from the HMD andto the base is computed as:

Φ_(HMD)=[Φ_(Φn)[0]+Φ_(xn),Φ_(Φn)[1]+φ_(yn),Φ_(Φn)[2]+Φ_(zn)]  (18)

resulting in a final beam position HF for transmitting sensor and otherdata from the HMD to the base at an incident angle Φ_(HMD) from initialbeam position H. As mentioned above, instead of determining the currentbeam position HF based directly from initial beam position H at startHMD position, a prior position already having its position relative tothe base and its beam position determined could be differenced from thecurrent HMD position data to determine the current HMD beam positionsinstead resulting in a series or chain of such computations as describedabove.

Process 800 may include “direct beam in direction of incident angle andtoward base” 816, where the beamforming or beam control unit on the HMDmay direct the beam to the base, and process 800 may include “transmitsensor data to base along beam” 818, and as described in detail above.Also as mentioned, this operation may involve steering the antenna atthe HMD to provide a transmitting beam position, but could alternativelyor additionally include steering the antenna at the base to form areceiving beam position.

Referring to FIG. 9, process 900 illustrates the operation of a sampleimage processing system 900 that performs a method of providing datafrom a remote base to a mobile device that displays images in real-time.In more detail, in the illustrated form, process 900 may include one ormore operations, functions or actions as illustrated by one or more ofactions 902 to 926 numbered evenly. By way of non-limiting example,process 900 will be described herein with reference to FIG. 10.Specifically, system 1000 includes logic units 1004 that have areal-time radio transmission beam direction control unit 1012. This unitmay have a base to mobile unit 1014 and a mobile to base unit 1020 thateach determine a beam position between a base and a mobile device. Oneexample of a mobile device is an HMD that receives real-time image datafrom the base and transmits information, such as sensor data form themobile device to the base. The run-time operation of such a system mayproceed as follows.

The process 900 may include “obtain sensed position and orientation dataof start position of mobile device” 902, and where the mobile device maybe an HMD with sensors as explained above, but could be other devicesthat display image data such as a smartphone to name one example. By oneform, such a smartphone is placed in headgear to form the display of anHMD. The sensors such as accelerometers, gyroscope, IMUs, opticalprojectors and receptors, RGBD cameras, and so forth described above maybe used to compute an (x, y, z) position of the mobile device as well asthe (i, j, k) orientation in degrees about the XYZ axes of the mobiledevice and from a zero orientation as described above. At this point,the (x, y, z) coordinates are likely to be relative to a 3D space of themobile device without an association to the position of the base. Alsoas mentioned, some or all of these sensor devices may be on the mobiledevice, on the base, both, some other location external to the mobiledevice and base, or any combination of these.

This operation also refers to receiving data after an initializationstage. During the initialization stage, radio-based transmission beamsmay be established between the mobile device and a base that has anantenna and beamforming ability, and can steer the beam toward themobile device. The transmission may be formed on a high bandwidth, lowlatency network, and by one example, a short-range wireless network suchas WiGig. The mobile device may be moved to at least two positions whereone of the positions is considered the start position. The angle of theestablished beam from the base to the mobile devices will be known bythe base. With these angles and the sensed 3D locations of the twomobile device positions, the distance between the two mobile devicepositions can be determined, and the position of the base relative tothe start position can be computed, and by triangulation in one example.This provides a sufficiently precise location of the start positionrelative to the base and in 3D space that may not be attainable withsensors alone. Thus, process 900 also may include “obtain data ofinitialized start position relative to the base” 904.

The process 900 may include “obtain sensed coordinates of the currentmobile position” 906, and as explained above, during the run-time of themobile device while a user is viewing images on the mobile device forexample, the sensors also provide the (x, y, z) coordinates and (i, j,k) orientation of the mobile device at the current position. The detailsare explained above.

The process 900 may include “determine change in position of mobiledevice relative to base” 908. Here, the difference between the start andcurrent sensed positions is determined for each x, y, z direction.Alternatively, the current position may be differenced from a priorposition of the mobile device whether the immediate prior position orsome other designated prior position in a series of mobile devicepositions, and where each mobile device position that is to be reliedupon to determine a current mobile device position relative to the basehas a position determined relative to the start position. The change inposition is added to the start or prior position being relied on todetermine the current position of the mobile device.

The computations here may be performed at the base or other remoteelectronic device or computer, and that transmits the results to thebase or mobile device. For the base example, the mobile device transmitssensor data to the base for computations over a short range wirelessnetwork such as WiGig, and the computations for change in position takeplace at the base. The base then uses the change to compute the beamangle where process 900 includes “determine base to mobile device beamangle” 910. By one example, this is by using trigonometric equation (10)described above.

By other alternatives, the change in position relative to the base aswell as the beam angle could be computed at the mobile device (oranother remote device) and then just the beam angle (or beam position)may be transmitted to the base for directing the beam from the base tothe mobile device.

The process 900 then may include “provide data to be transmitted fromthe base to mobile device” 912, and this may include providing imagedata to be displayed at the mobile device and to perform whateveroperations are needed to perform the transmission of the image to themobile device, such as encoding of the data.

To establish a transmission beam from the mobile device to the base inorder to transmit sensor data to the base for the computations performedat the base mentioned above, the process 900 may include “transmit datato mobile device to determine incident angle of current mobile devicerelative to start or prior beam position” 914. As described above, theincident angle is the angle from the initial beam position to a beamposition directed from the current position of the mobile device to thebase. The data to be transmitted to the mobile device may include thecomputed change in position. The current sensed orientation data may betransmitted as well when the sensed orientation is not computed at themobile device itself. This transmission is performed when the mobiledevice is to compute the beam position to transmit sensor data to thebase.

Alternatively, however, the transmission may be omitted when theincident angle is computed at the base instead. In this case, just theincident angle may be transmitted from the base and to the mobile devicefor the mobile device to set the beam position for the sensor datatransmission. By yet another alternative, no transmission is needed whenthe base steers the radio beam to a receiving radio beam position toreceive the sensor data from the mobile device. Other variations may beapplied as well as long as radio-based searches are not used to set beamposition for data transmission after the initialization stage and untilthe setting of the start position of the mobile device relative to thebase is to be refreshed or recomputed.

The process 900 may include “determine a position component angle usingthe change in position relative to the base” 916, and by one example,this includes setting the beam position from the base to the startposition of the mobile device as the x-axis of a new xyz coordinatesystem. The system then determines the current position of the mobiledevice on this new coordinate system using the difference in sensedposition from the start position to the current position. Trigonometricequations can then be used to determine x, y, and z sub-component anglesof the difference in angle from the initial beam position (at the startmobile device position) to a line from the base to the current HMDposition, and which is the position component angle of the incidentangle. Each sub-component refers to an angle about the referenced axis(x, y, or z) and in the plane of the other two axes. The details areexplained above. It also will be appreciated that instead of using theinitial angle, the incident angle could be established as a differenceto any other prior beam position established with sufficient accuracywhether during an initialization period or as a refresh of the initialbeam position, and whether or not established by radio-basedtransmission beam steering search or by sensors, when sufficientlyprecise.

The process 900 may include “determine an orientation component angle byusing the change in orientation” 918. Here, the (i, j, k) orientationsof the mobile device at the current position and the start (or prior)position are differenced with each sub-component referencing the sameangle being subtracted (e.g., i₁−i_(n) where both are about the x-axis).

The process 900 may include “determine a total incident angle from themobile device to the base” 920, and as described above for equation(18), where the position component and orientation component about thesame axis are added together to determine a total incident angle indegrees about each axis.

The process 900 may include “determine a beam position based on theincident angle” 922, and the system then sets the beam position totransmit sensor or other data from the mobile device to the base along aline at the incident angle from the initial (or prior) beam position.Other details are provided above.

The process 900 may include “transmit data from the mobile device to thebase” 924, where the mobile device transmits the data over ashort-range, high bandwidth, low latency wireless network such as WiGigby one example, although other wireless networks could be used here aswell.

The process 900 may include the query “more beam positions to update?”,and if so the process loops back to operation 902 to sense the newcurrent mobile device position. This may be triggered by motion of themobile device, or may be performed at certain time intervals whether ornot the mobile device has been moved.

It will be appreciated that the processes 300, 400, 700, 800, and 900respectively explained with FIGS. 3-4 and 7-9 do not necessarily need tobe performed in the order shown, nor with all of the operations shown.It will be understood that some operations may be skipped or performedin different orders.

Also, any one or more of the operations of FIGS. 3-4 and 7-9 may beundertaken in response to instructions provided by one or more computerprogram products. Such program products may include signal bearing mediaproviding instructions that, when executed by, for example, a processor,may provide the functionality described herein. The computer programproducts may be provided in any form of one or more machine-readablemedia. Thus, for example, a processor including one or more processorcore(s) may undertake one or more of the operations of the exampleprocesses herein in response to program code and/or instructions orinstruction sets conveyed to the processor by one or more computer ormachine-readable media. In general, a machine-readable medium may conveysoftware in the form of program code and/or instructions or instructionsets that may cause any of the devices and/or systems to perform asdescribed herein. The machine or computer readable media may be anon-transitory article or medium, such as a non-transitory computerreadable medium, and may be used with any of the examples mentionedabove or other examples except that it does not include a transitorysignal per se. It does include those elements other than a signal per sethat may hold data temporarily in a “transitory” fashion such as RAM andso forth.

As used in any implementation described herein, the term “module” refersto any combination of software logic, firmware logic and/or hardwarelogic configured to provide the functionality described herein. Thesoftware may be embodied as a software package, code and/or instructionset or instructions, and “hardware”, as used in any implementationdescribed herein, may include, for example, singly or in anycombination, hardwired circuitry, programmable circuitry, state machinecircuitry, and/or firmware that stores instructions executed byprogrammable circuitry. The modules may, collectively or individually,be embodied as circuitry that forms part of a larger system, forexample, an integrated circuit (IC), system on-chip (SoC), and so forth.For example, a module may be embodied in logic circuitry for theimplementation via software, firmware, or hardware of the coding systemsdiscussed herein.

As used in any implementation described herein, the term “logic unit”refers to any combination of firmware logic and/or hardware logicconfigured to provide the functionality described herein. The logicunits may, collectively or individually, be embodied as circuitry thatforms part of a larger system, for example, an integrated circuit (IC),system on-chip (SoC), and so forth. For example, a logic unit may beembodied in logic circuitry for the implementation firmware or hardwareof the coding systems discussed herein. One of ordinary skill in the artwill appreciate that operations performed by hardware and/or firmwaremay alternatively be implemented via software, which may be embodied asa software package, code and/or instruction set or instructions, andalso appreciate that logic unit may also utilize a portion of softwareto implement its functionality.

As used in any implementation described herein, the term “component” mayrefer to a module or to a logic unit, as these terms are describedabove. Accordingly, the term “component” may refer to any combination ofsoftware logic, firmware logic, and/or hardware logic configured toprovide the functionality described herein. For example, one of ordinaryskill in the art will appreciate that operations performed by hardwareand/or firmware may alternatively be implemented via a software module,which may be embodied as a software package, code and/or instructionset, and also appreciate that a logic unit may also utilize a portion ofsoftware to implement its functionality.

Referring to FIG. 10, an example image processing system 1000 isarranged in accordance with at least some implementations of the presentdisclosure. In various implementations, the example image processingsystem 1000 may perform many of the functions of determining radiotransmission beam positions described above. Thus, system 1000 may bethe mobile device such as an HMD, the base wirelessly communicating withthe mobile device, both (where one system 1000 may be the mobile deviceand another system 1000 may be the base), or some other electronicdevice communicating with the mobile device and/or the base. When thesystem 1000 is the mobile device, the system 1000 has at least thoseunits shown to receive image data and transmit sensor data. Thus, inaddition to the HMD, the system 1000 may be a multi-function device suchas a smartphone, smart glasses, tablet, and so forth.

When the system 1000 is the base, the system 1000 has at least thoseunits shown for transmitting image data and receiving sensor data from amobile device. In these cases, the system 1000, or at least the part ofthe image processing system 1000 that holds logic units 1004 thatperforms beam position computations, may be a desktop or laptopcomputer, smartphone, tablet, or other device. It also could be or havea fixed function device such as a set top box (cable box or satellitebox), game box, or a television.

When the system 1000 is some other electronic device, the system 1000,or at least the logic units 1004 of system 1000, may perform the beamforming computations for the mobile device, the base, or both. In thisexample case, the system 1000 may wirelessly (or by wire) communicatedata to the mobile device or base but is not by the same wirelessnetwork used between the base and mobile device that uses radiotransmission beam steering. By alternative approaches, the system 1000could still have this capability. In these cases, the system 1000 may bea remote server, computer, laptop, or one of the sensors used by thebase or mobile device such as a digital camera whether a dedicatedcamera or a camera on a multi-function device.

Otherwise, when the system 1000 is not itself a camera, the system 1000may be considered to have imaging device(s) 1002. In any of these cases,such technology may include a camera such as a digital camera system, adedicated camera device, web cam, or any other device with a camera tobe an external still or video camera capturing the user wearing themobile device and the area around the user for example, or capturing theenvironment surrounding the mobile device when such a camera is on themobile device. The camera may be an RGB camera or an RGB-D camera, butcould be a YUV camera. The internal camera may be an RGB or YUV colorcamera, monochrome camera, or an IR camera with a projector and sensor.Thus, in one form, image processing system 1000 may include the imagingdevice(s) 1002 and that has camera hardware, camera software, units,modules, components, and optics including one or more sensors as well asauto-focus, zoom, aperture, ND-filter, auto-exposure, flash, actuatorcontrols, and so forth. In other cases, the imaging device(s) 1002 maybe considered physically remote from the rest of the image processingsystem 1000, and may be wirelessly communicating, or wired tocommunicate, image data to the logic units 1004. In this form, logicmodules 1004 may communicate remotely with, or otherwise may becommunicatively coupled to, the imaging device 1002 for furtherprocessing of the image data.

The system 1000 also may have other sensor device(s) 1006 which mayinclude those sensors at the mobile device, base, or other electronicdevice processing data to determine beam positions, or may be remotefrom any of these when the sensor is self-contained. When the mobiledevice is an HMD the sensor data may be used to determine theperspective of the images to be displayed on the HMD as well. Suchsensors may include one or more gyroscopes, accelerometers, IMUs, GPSs,optical projector systems such whether IR, RGB, or other lighttechnology, and so forth.

The logic modules 1004 of the image processing system 1000 may have asensor unit 1008 that receives sensor data, which may or may not be rawsensor data, and may process the sensor data into a format expected forimage processing and/or generation as well as beam positioncomputations. An image generation unit 1010 may use the sensor data toprocess captured images to create a 3D model or generate animatedimages. Thus, the image generation unit 1010 may perform pre-processing,decoding, encoding, and/or even post-processing to prepare the imagedata for transmission, storage, and/or display.

In the illustrated example, the logic modules 1004 also may include areal-time radio transmission beam direction control unit 1012. Thecontrol unit 1012 may have an initialization unit 1014, base to mobileunit 1016, beam steering unit 1018, and a mobile to base unit 1020 whichincludes a position angle unit 1022 and an orientation angle unit 1024.These units may be used to determine a beam position and have similarfunctions as those units described above with similar titles such asthose on system 200. The logic units 1004 also may have a beamformingunit 1026 to direct the beam and image display unit 1028 that controlsthe display at least when the system 1000 is, or has, a mobile device.

These units may be operated by, or even entirely or partially locatedat, one or more processor(s) 1040, and which may include an image signalprocessor (ISP) 1042 to perform many of the operations mentioned herein.The image signal processor (ISP) 1042 may be an Intel Atom by oneexample. The system 1000 also may have memory stores 1044 which may beRAM, DRAM, DDR RAM, cache, or other memory types, and which may or maynot hold the image data and statistics as well as the logic unitsmentioned above. The system 1000 also may have at least one antenna 1038that is steerable for beamforming as described above, and one or moreantennas may be provided whether the system 1000 is a mobile device, abase, or other device mentioned above. The antenna may be used totransmit and/or receive sensor data, image data, or overhead data, orother data. The sensors may use these antennas or may use other antennasto perform sensing operations.

In one example implementation, the image processing system 1000 may havea display 1046, which may or may not be one or more displays on the HMD,at least one of the processor(s) 1040 communicatively coupled to thedisplay, and at least one memory 1044 communicatively coupled to theprocessor to perform the operations described herein as explained above.The image generation unit 1010, which may have an encoder and decoder,and antenna 1038 may be provided to transmit compressed image date toand from other devices that may display or store the images. This mayrefer to transmission of image data between the base and the mobiledevice no matter which of the devices has the logic units 1004.Otherwise, the processed image 1048 may be displayed on the display 1046or stored in memory 1044. As illustrated, any of these components may becapable of communication with one another and/or communication withportions of logic modules 1004 and/or imaging device 1002. Thus,processors 1040 may be communicatively coupled to both the image device1002 and the logic modules 1004 for operating those components. By oneapproach, although image processing system 1000, as shown in FIG. 10,may include one particular set of unit or actions associated withparticular components or modules, these units or actions may beassociated with different components or modules than the particularcomponent or module illustrated here.

Referring to FIG. 11, an example system 1100 in accordance with thepresent disclosure operates one or more aspects of the image processingsystem described herein. It will be understood from the nature of thesystem components described below that such components may be associatedwith, or used to operate, certain part or parts of the image processingsystems described above including performance of a mobile device such asan HMD or other mobile device with a display for virtual or augmentedreality generation for example, and/or operation of the base describedabove. In various implementations, system 1100 may be a media systemalthough system 1100 is not limited to this context. For example, system1100 may be incorporated into a digital video camera, mobile device withcamera or video functions such as an imaging phone, webcam, personalcomputer (PC), remote server, laptop computer, ultra-laptop computer,tablet, touch pad, portable computer, handheld computer, palmtopcomputer, personal digital assistant (PDA), cellular telephone,combination cellular telephone/PDA, television, smart device (e.g.,smart phone, smart tablet or smart television), mobile internet device(MID), messaging device, data communication device, and so forth.

In various implementations, system 1100 includes a platform 1102 coupledto a display 1120. Platform 1102 may receive content from a contentdevice such as content services device(s) 1130 or content deliverydevice(s) 1140 or other similar content sources. A navigation controller1150 including one or more navigation features may be used to interactwith, for example, platform 1102 and/or display 1120. Each of thesecomponents is described in greater detail below.

In various implementations, platform 1102 may include any combination ofa chipset 1105, processor 1110, memory 1112, storage 1114, graphicssubsystem 1115, applications 1116 and/or radio 1118. Chipset 1105 mayprovide intercommunication among processor 1110, memory 1112, storage1114, graphics subsystem 1115, applications 1116 and/or radio 1118. Forexample, chipset 1105 may include a storage adapter (not depicted)capable of providing intercommunication with storage 1114.

Processor 1110 may be implemented as a Complex Instruction Set Computer(CISC) or Reduced Instruction Set Computer (RISC) processors; x86instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In variousimplementations, processor 1110 may be dual-core processor(s), dual-coremobile processor(s), and so forth.

Memory 1112 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM).

Storage 1114 may be implemented as a non-volatile storage device suchas, but not limited to, a magnetic disk drive, optical disk drive, tapedrive, an internal storage device, an attached storage device, flashmemory, battery backed-up SDRAM (synchronous DRAM), and/or a networkaccessible storage device. In various implementations, storage 1114 mayinclude technology to increase the storage performance enhancedprotection for valuable digital media when multiple hard drives areincluded, for example.

Graphics subsystem 1115 may perform processing of images such as stillor video for display. Graphics subsystem 1115 may be a graphicsprocessing unit (GPU) or a visual processing unit (VPU), for example. Ananalog or digital interface may be used to communicatively couplegraphics subsystem 1115 and display 1120. For example, the interface maybe any of a High-Definition Multimedia Interface, Display Port, wirelessHDMI, and/or wireless HD compliant techniques. Graphics subsystem 1115may be integrated into processor 1110 or chipset 1105. In someimplementations, graphics subsystem 1115 may be a stand-alone cardcommunicatively coupled to chipset 1105.

The graphics and/or video processing techniques described herein may beimplemented in various hardware architectures. For example, graphicsand/or video functionality may be integrated within a chipset.Alternatively, a discrete graphics and/or video processor may be used.As still another implementation, the graphics and/or video functions maybe provided by a general purpose processor, including a multi-coreprocessor. In further implementations, the functions may be implementedin a consumer electronics device.

Radio 1118 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), personalarea networks (PANs), cellular networks, and satellite networks. Incommunicating across such networks, radio 1118 may operate in accordancewith one or more applicable standards in any version, and may beoperated in conjunction with the antenna described below.

In various implementations, display 1120 may include any television typemonitor or display. Display 1120 may include, for example, a computerdisplay screen, touch screen display, video monitor, television-likedevice, and/or a television. Display 1120 may be digital and/or analog.The display 1120 also may be a display on an HMD as described above. Invarious implementations, display 1120 may be a holographic display.Also, display 1120 may be a transparent surface that may receive avisual projection. Such projections may convey various forms ofinformation, images, and/or objects. For example, such projections maybe a visual overlay for a mobile augmented reality (MAR) application.Under the control of one or more software applications 1116, platform1102 may display user interface 1122 on display 1120.

In various implementations, content services device(s) 1130 may behosted by any national, international and/or independent service andthus accessible to platform 1102 via the Internet, for example. Contentservices device(s) 1130 may be coupled to platform 1102 and/or todisplay 1120. Platform 1102 and/or content services device(s) 1130 maybe coupled to a network 1160 to communicate (e.g., send and/or receive)media information to and from network 1160. Content delivery device(s)1140 also may be coupled to platform 1102 and/or to display 1120.

In various implementations, content services device(s) 1130 may includea cable television box, personal computer, network, telephone, Internetenabled devices or appliance capable of delivering digital informationand/or content, and any other similar device capable of unidirectionallyor bidirectionally communicating content between content providers andplatform 1102 and/display 1120, via network 1160 or directly. It will beappreciated that the content may be communicated unidirectionally and/orbidirectionally to and from any one of the components in system 1100 anda content provider via network 1160. Examples of content may include anymedia information including, for example, video, music, medical andgaming information, and so forth.

Content services device(s) 1130 may receive content such as cabletelevision programming including media information, digital information,and/or other content. Examples of content providers may include anycable or satellite television or radio or Internet content providers.The provided examples are not meant to limit implementations inaccordance with the present disclosure in any way.

In various implementations, platform 1102 may receive control signalsfrom navigation controller 1150 having one or more navigation features.The navigation features of controller 1150 may be used to interact withuser interface 1122, for example. In implementations, navigationcontroller 1150 may be a pointing device that may be a computer hardwarecomponent (specifically, a human interface device) that allows a user toinput spatial (e.g., continuous and multi-dimensional) data into acomputer. Many systems such as graphical user interfaces (GUI), andtelevisions and monitors allow the user to control and provide data tothe computer or television using physical gestures.

Movements of the navigation features of controller 1150 may bereplicated on a display (e.g., display 1120) by movements of a pointer,cursor, focus ring, or other visual indicators displayed on the display.For example, under the control of software applications 1116, thenavigation features located on navigation controller 1150 may be mappedto virtual navigation features displayed on user interface 1122, forexample. In implementations, controller 1150 may not be a separatecomponent but may be integrated into platform 1102 and/or display 1120.The present disclosure, however, is not limited to the elements or inthe context shown or described herein.

In various implementations, drivers (not shown) may include technologyto enable users to instantly turn on and off platform 1102 like atelevision with the touch of a button after initial boot-up, whenenabled, for example. Program logic may allow platform 1102 to streamcontent to media adaptors or other content services device(s) 1130 orcontent delivery device(s) 1140 even when the platform is turned “off.”In addition, chipset 1105 may include hardware and/or software supportfor 8.1 surround sound audio and/or high definition (7.1) surround soundaudio, for example. Drivers may include a graphics driver for integratedgraphics platforms. In implementations, the graphics driver may comprisea peripheral component interconnect (PCI) Express graphics card.

In various implementations, any one or more of the components shown insystem 1100 may be integrated. For example, platform 1102 and contentservices device(s) 1130 may be integrated, or platform 1102 and contentdelivery device(s) 1140 may be integrated, or platform 1102, contentservices device(s) 1130, and content delivery device(s) 1140 may beintegrated, for example. In various implementations, platform 1102 anddisplay 1120 may be an integrated unit. Display 1120 and content servicedevice(s) 1130 may be integrated, or display 1120 and content deliverydevice(s) 1140 may be integrated, for example. These examples are notmeant to limit the present disclosure.

In various implementations, system 1100 may be implemented as a wirelesssystem, but also may include wired systems. When implemented as awireless system, system 1100 may include components and interfacessuitable for communicating over a wireless shared media, such as one ormore antennas, transmitters, receivers, transceivers, amplifiers,filters, control logic, and so forth. An example of wireless sharedmedia may include portions of a wireless spectrum, such as the RFspectrum and so forth. When implemented as a wired system, system 1100may include components and interfaces suitable for communicating overwired communications media, such as input/output (I/O) adapters,physical connectors to connect the I/O adapter with a correspondingwired communications medium, a network interface card (NIC), disccontroller, video controller, audio controller, and the like. Examplesof wired communications media may include a wire, cable, metal leads,printed circuit board (PCB), backplane, switch fabric, semiconductormaterial, twisted-pair wire, co-axial cable, fiber optics, and so forth.

Platform 1102 may establish one or more logical or physical channels tocommunicate information. The information may include media informationand control information. Media information may refer to any datarepresenting content meant for a user. Examples of content may include,for example, data from a voice conversation, videoconference, streamingvideo, electronic mail (“email”) message, voice mail message,alphanumeric symbols, graphics, image, video, text and so forth. Datafrom a voice conversation may be, for example, speech information,silence periods, background noise, comfort noise, tones and so forth.Control information may refer to any data representing commands,instructions or control words meant for an automated system. Forexample, control information may be used to route media informationthrough a system, or instruct a node to process the media information ina predetermined manner. The implementations, however, are not limited tothe elements or in the context shown or described in FIG. 11.

Referring to FIG. 12, a small form factor device 1200 is one example ofthe varying physical styles or form factors in which systems 1000 or1100 may be embodied. By this approach, device 1200 may be implementedas a mobile computing device having wireless capabilities, and by oneexample, could be used as a screen in an HMD. A mobile computing devicemay refer to any device having a processing system and a mobile powersource or supply, such as one or more batteries, for example.

As described above, examples of a mobile computing device may include ana CR or AR HMD, as well as a digital still camera, digital video camera,mobile devices with camera or video functions such as imaging phones,webcam, personal computer (PC), laptop computer, ultra-laptop computer,tablet, touch pad, portable computer, handheld computer, palmtopcomputer, personal digital assistant (PDA), cellular telephone,combination cellular telephone/PDA, television, smart device (e.g.,smart phone, smart tablet or smart television), mobile internet device(MID), messaging device, data communication device, and so forth.

Examples of a mobile computing device also may include computers thatare arranged to be worn by a person, such as a wrist computer, fingercomputer, ring computer, eyeglass computer, belt-clip computer, arm-bandcomputer, shoe computers, clothing computers, and other wearablecomputers. In various embodiments, for example, a mobile computingdevice may be implemented as a smart phone capable of executing computerapplications, as well as voice communications and/or datacommunications. Although some embodiments may be described with a mobilecomputing device implemented as a smart phone by way of example, it maybe appreciated that other embodiments may be implemented using otherwireless mobile computing devices as well. The implementations are notlimited in this context.

As shown in FIG. 12, device 1200 may include a housing with a front 1201and a back 1202. Device 1200 includes a display 1204, an input/output(I/O) device 1206, and an integrated antenna 1208. Device 1200 also mayinclude navigation features 1212. I/O device 1206 may include anysuitable I/O device for entering information into a mobile computingdevice. Examples for I/O device 1206 may include an alphanumerickeyboard, a numeric keypad, a touch pad, input keys, buttons, switches,microphones, speakers, voice recognition device and software, and soforth. Information also may be entered into device 1200 by way ofmicrophone 1214, or may be digitized by a voice recognition device. Asshown, device 1200 may include a camera 1205 (e.g., including at leastone lens, aperture, and imaging sensor) and a flash 1210 integrated intoback 1202 (or elsewhere) of device 1200. The implementations are notlimited in this context.

Various forms of the devices and processes described herein may beimplemented using hardware elements, software elements, or a combinationof both. Examples of hardware elements may include processors,microprocessors, circuits, circuit elements (e.g., transistors,resistors, capacitors, inductors, and so forth), integrated circuits,application specific integrated circuits (ASIC), programmable logicdevices (PLD), digital signal processors (DSP), field programmable gatearray (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

While certain features set forth herein have been described withreference to various implementations, this description is not intendedto be construed in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to lie within the spirit and scope of thepresent disclosure.

The following examples pertain to further implementations.

By one example, a computer-implemented method of image processingcomprises obtaining image sensing a 3D position of at least one headmounted display (HMD) arranged to display video sequences to a person,determining the location of the sensed 3D HMD position relative to abase, wherein the sensing and determining is performed with other than aradio-based search; determining a beam position of a wirelessradio-based transmission beam by using the 3D HMD position; andwirelessly transmitting images from the base and directed toward the HMDalong the beam position to display the images to the person at the HMD.

By another implementation, the method may include that the 3D HMDposition is a start HMD position, the method comprising: moving the HMDto at least one subsequent current HMD position from the start HMDposition causing a change in HMD position; determining the measure ofthe change in HMD position so that the current HMD position indicates aposition relative to the base; determining an updated beam positiondirected to the current HMD position by using the measure of the changein HMD position. sensing a series of changes from prior HMD positions tomultiple current HMD positions; determining the HMD position ofindividual ones of the current HMD positions and relative to the startHMD position to update the beam position directed to the individualcurrent HMD positions; sensing a series of changes from prior HMDpositions to multiple current HMD positions; determining the HMDposition of an individual one of the current HMD positions relative to aprior HMD position in the series and to update the beam positiondirected to the individual one current HMD position; determining a firstbeam position while the HMD is held in a first location; determining theposition of the HMD and relative to the first beam position; moving theHMD to at least one other position; determining the position of the beamat the other position(s); and using at least two of the beam positionsand the sensed HMD positions to triangulate a position of the baserelative to the HMD at the first location before using sensor data todetermine the position of the beam at additional HMD positions. Themethod including that the sensors for locating the 3D position of theHMD are on the HMD or at least some of the sensors for locating the 3Dposition of the HMD are on the base or on both the HMD and the base.

The method also comprising transmitting sensor data from sensors and tothe base to be used to compute, at the base, the 3D HMD position and thebeam position, wherein at least some of the sensors used to compute the3D HMD position are on the HMD and generate sensor data at the HMD, themethod comprising computing at the HMD, both the 3D position of the HMDand a beam position to transmit the sensor data from the HMD to thebase. The method also comprises generating the image with a perspectivethat corresponds to the location and orientation of the HMD by usingsensor data also used to determine the 3D HMD position; the methodcomprising at least one of: (1) wherein a beam position is determinedusing non-radio-based search sensors each time the HMD is moved apredetermined minimum amount, and (2) wherein a beam position isdetermined using non-radio-based search sensors each predetermined timeinterval regardless of the motion of the HMD. The method may comprisewherein a wireless network forming the transmissions has a frequencyband and bandwidth sufficient to provide video images on the HMD andperceived to be in near real-time without substantial delay noticeableto a user and while the HMD is moving, wherein the network is WiGig.

By one approach, a head mounted display comprises a body; at least onesensor at the body and to sense a 3D HMD position of the head mounteddisplay (HMD); a beam direction generation unit to determine a beamposition by determining the location of the 3D HMD position relative toa base, wherein the sensor and beam direction generation unitrespectively sense and determine the location of the 3D HMD byperforming other than a radio-based search; a receiver to wirelesslyreceive transmitted images along a wireless radio-based transmissionbeam aimed toward the receiver along the beam position; and a display atthe body to display the images at the HMD and to a person wearing theHMD.

By another form, the head mounted display includes wherein the beamposition is established at the base; the beam generation unit to use the3D HMD position to determine a second beam position aimed from the HMDand toward the base; and a transmitter to transmit sensor data of the atleast one sensor along a beam at the second beam position and to thebase, wherein the beam generation unit to generate an incident anglecomprising factoring (1) a position angle measuring a change in anglerelative to the base from a prior HMD position to a subsequent HMDposition, and (2) an orientation angle measuring the change inorientation of the HMD in the prior HMD position and in the subsequentHMD position; and directing the second beam position along the incidentangle.

By one form, a computer-implemented system comprises a head mounteddisplay (HMD) having a body and a display on the body to display imagesat the HMD and to a person wearing the HMD; a base remote from the HMDand having a transmitter for wirelessly transmitting images to the HMDalong a radio-based transmission beam aimed at the HMD; at least onesensor at or communicating with the HMD, base, or both to sense a 3D HMDposition or orientation or both of the head mounted display (HMD) and atleast when the HMD has been moved; a beam generating unit at the HMD orbase or both and that uses the 3D HMD position to determine a positionof the beam to extend between the base and the HMD, wherein the sensingand determining is performed by other than a radio-based search.

By yet another implementation, the computer-implemented system compriseswherein the HMD comprises: a beam generating unit to determine theposition of a second beam position without the use of a radio-basedsearch; and a transmitter to transmit the sensor data to the HMD alongthe second beam position. The beam generation unit is to: determine anincident angle factoring a change in angle associated with a change inHMD position and a change in orientation between the HMD at the priorHMD position and the at least one subsequent HMD position; and directthe second beam position along the incident angle toward the base,wherein a wireless network forming the transmission and beam is anetwork with at least the transmission bandwidth of WiGig.

By one approach, at least one computer readable article comprises aplurality of instructions that in response to being executed on acomputing device, cause the computing device to operate by: sensing aposition or orientation or both of a mobile device, wherein the sensingis other than a radio-based search; determining a beam position of awireless radio transmission beam and from a remote transmitter base andtoward the mobile device by using the sensed position or orientation orboth of the mobile device and without performing a radio-based search;and wirelessly transmitting data between the transmitter base and themobile device along the beam position to perform functions at the mobiledevice or the transmitting base or both using the transmitted data.

By another approach, the instructions cause the computing device to beoperated by wherein the mobile device is a display device to displayimages of a video sequence, wherein the transmitted data forms imageshaving a perspective determined by using sensor data from the sensing ofthe position of the mobile device, and wherein a wireless networkforming the transmissions has a frequency band and bandwidth sufficientto provide video images on the display device and perceived to be innear real-time without substantial delay noticeable to a user and whilethe display device is moving, wherein the instructions cause thecomputing device to operate by: moving the HMD to at least one HMDposition or orientation or both from a prior HMD position or orientationor both; determining the change in HMD position or orientation or both;and determining an updated beam position associated with the moving ofthe HMD comprising using a measure of the change in HMD position ororientation or both.

In a further example, at least one machine readable medium may include aplurality of instructions that in response to being executed on acomputing device, causes the computing device to perform the methodaccording to any one of the above examples.

In a still further example, an apparatus may include means forperforming the methods according to any one of the above examples.

The above examples may include specific combination of features.However, the above examples are not limited in this regard and, invarious implementations, the above examples may include undertaking onlya subset of such features, undertaking a different order of suchfeatures, undertaking a different combination of such features, and/orundertaking additional features than those features explicitly listed.For example, all features described with respect to any example methodsherein may be implemented with respect to any example apparatus, examplesystems, and/or example articles, and vice versa.

What is claimed is:
 1. A computer-implemented method of imageprocessing, comprising: sensing a 3D position of at least one headmounted display (HMD) arranged to display video sequences to a person,determining the location of the sensed 3D HMD position relative to abase, wherein the sensing and determining is performed with other than aradio-based search; determining a beam position of a wirelessradio-based transmission beam by using the 3D HMD position; andwirelessly transmitting images from the base and directed toward the HMDalong the beam position to display the images to the person at the HMD.2. The method of claim 1 wherein the 3D HMD position is a start HMDposition, the method comprising: moving the HMD to at least onesubsequent current HMD position from the start HMD position causing achange in HMD position; determining the measure of the change in HMDposition so that the current HMD position indicates a position relativeto the base; and determining an updated beam position directed to thecurrent HMD position by using the measure of the change in HMD position.3. The method of claim 2 comprising: sensing a series of changes fromprior HMD positions to multiple current HMD positions; determining theHMD position of individual ones of the current HMD positions andrelative to the start HMD position to update the beam position directedto the individual current HMD positions.
 4. The method of claim 2comprising: sensing a series of changes from prior HMD positions tomultiple current HMD positions; determining the HMD position of anindividual one of the current HMD positions relative to a prior HMDposition in the series and to update the beam position directed to theindividual one current HMD position.
 5. The method of claim 1,comprising: determining a first beam position while the HMD is held in afirst location; determining the position of the HMD and relative to thefirst beam position; moving the HMD to at least one other position;determining the position of the beam at the other position(s); and usingat least two of the beam positions and the sensed HMD positions totriangulate a position of the base relative to the HMD at the firstlocation before using sensor data to determine the position of the beamat additional HMD positions.
 6. The method of claim 1 wherein thesensors for locating the 3D position of the HMD are on the HMD.
 7. Themethod of claim 1 wherein at least some of the sensors for locating the3D position of the HMD are on the base or on both the HMD and the base.8. The method of claim 1 comprising transmitting sensor data fromsensors and to the base to be used to compute, at the base, the 3D HMDposition and the beam position.
 9. The method of claim 1 wherein atleast some of the sensors used to compute the 3D HMD position are on theHMD and generate sensor data at the HMD, the method comprising computingat the HMD, both the 3D position of the HMD and a beam position totransmit the sensor data from the HMD to the base.
 10. The method ofclaim 1 comprising: generating the image with a perspective thatcorresponds to the location and orientation of the HMD by using sensordata also used to determine the 3D HMD position.
 11. The method of claim1 wherein a beam position is determined using non-radio-based searchsensors each time the HMD is moved a predetermined minimum amount. 12.The method of claim 1 wherein a beam position is determined usingnon-radio-based search sensors each predetermined time intervalregardless of the motion of the HMD.
 13. The method of claim 1 wherein awireless network forming the transmissions has a frequency band andbandwidth sufficient to provide video images on the HMD and perceived tobe in near real-time without substantial delay noticeable to a user andwhile the HMD is moving.
 14. The method of claim 13 wherein the networkis WiGig.
 15. A head mounted display, comprising: a body; at least onesensor at the body and to sense a 3D HMD position of the head mounteddisplay (HMD); a beam direction generation unit to determine a beamposition by determining the location of the 3D HMD position relative toa base, wherein the sensor and beam direction generation unitrespectively sense and determine the location of the 3D HMD byperforming other than a radio-based search; a receiver to wirelesslyreceive transmitted images along a wireless radio-based transmissionbeam aimed toward the receiver along the beam position; and a display atthe body to display the images at the HMD and to a person wearing theHMD.
 16. The head mounted display of claim 15 wherein the beam positionis established at the base.
 17. The head mounted display of claim 15comprising: the beam generation unit to use the 3D HMD position todetermine a second beam position aimed from the HMD and toward the base;and a transmitter to transmit sensor data of the at least one sensoralong a beam at the second beam position and to the base.
 18. The headmounted display of claim 17 wherein the beam generation unit to generatean incident angle comprising factoring (1) a position angle measuring achange in angle relative to the base from a prior HMD position to asubsequent HMD position, and (2) an orientation angle measuring thechange in orientation of the HMD in the prior HMD position and in thesubsequent HMD position; and directing the second beam position alongthe incident angle.
 19. An image processing system, comprising: a headmounted display (HMD) having a body and a display on the body to displayimages at the HMD and to a person wearing the HMD; a base remote fromthe HMD and having a transmitter for wirelessly transmitting images tothe HMD along a radio-based transmission beam aimed at the HMD; at leastone sensor at or communicating with the HMD, base, or both to sense a 3DHMD position or orientation or both of the head mounted display (HMD)and at least when the HMD has been moved; a beam generating unit at theHMD or base or both and that uses the 3D HMD position to determine aposition of the beam to extend between the base and the HMD, wherein thesensing and determining is performed by other than a radio-based search.20. The system of claim 19, wherein the HMD comprises: a beam generatingunit to determine the position of a second beam position without the useof a radio-based search; and a transmitter to transmit the sensor datato the HMD along the second beam position.
 21. The system of claim 20comprising: the beam generation unit to: determine an incident anglefactoring a change in angle associated with a change in HMD position anda change in orientation between the HMD at the prior HMD position andthe at least one subsequent HMD position; and direct the second beamposition along the incident angle toward the base.
 22. The system ofclaim 19 wherein a wireless network forming the transmission and beam isa network with at least the transmission bandwidth of WiGig.
 23. Atleast one computer readable article comprising a plurality ofinstructions that in response to being executed on a computing device,causes the computing device to operate by: sensing a position ororientation or both of a mobile device, wherein the sensing is otherthan a radio-based search; determining a beam position of a wirelessradio transmission beam and from a remote transmitter base and towardthe mobile device by using the sensed position or orientation or both ofthe mobile device and without performing a radio-based search; andwirelessly transmitting data between the transmitter base and the mobiledevice along the beam position to perform functions at the mobile deviceor the transmitting base or both using the transmitted data.
 24. Thearticle of claim 23 wherein the mobile device is a display device todisplay images of a video sequence, wherein the transmitted data formsimages having a perspective determined by using sensor data from thesensing of the position of the mobile device, and wherein a wirelessnetwork forming the transmissions has a frequency band and bandwidthsufficient to provide video images on the display device and perceivedto be in near real-time without substantial delay noticeable to a userand while the display device is moving.
 25. The article of claim 24wherein the instructions cause the computing device to operate by:moving the HMD to at least one HMD position or orientation or both froma prior HMD position or orientation or both; determining the change inHMD position or orientation or both; and determining an updated beamposition associated with the moving of the HMD comprising using ameasure of the change in HMD position or orientation or both.